- - Knowledge base overview
Product Updates

Home
Community overview

The Striim Community

Featured Topics

Recently active
Help others

S

sriram9780New Participant

Product Q&A

How to retrieve last checkpoint after Source Component has been updated?

For a CDC app using MySQL reader, I use the last checkpoint of the source component using Mon <namespace>.<application name> before I restart an app.After making a change to the source component , the checkpoint info is gone.Is there any other way I can retrieve the last checkpoint?Any tips are deeply appreciated and also curious to see different options available for these kind of scenarios. Thanks in advance!

60

S

20 days ago

emiliogabriel000New Participant

Product Q&A

MySQL data migration to alloydb

Hello, I am participating in a project at the company I work for where we are considering migrating data from a MYSQL database to ALLOYDB but so far I have not found any effective documentation that could help me... could you give me some support with this?

abgNew Participant

Error "Please upload a local file" when uploading MySQL driver in Developer Edition

I receive the Error "Please upload a local file" when I try to upload MySQL driver in the Developer Edition.I have tried to upload MySQL and MariaDB drivers and I always receive the same error.

404

4 months ago

muraliNew Participant

Community Q&A

Removing Square brackets from JSON message before publishing to Pubsub

We are publishing CDC changes from Oracle table to GBQ PubSub, however the Striim is publishing JSON Array with square brackets with one CDC change. I don’t want those flower brackets , how can we resolve this. Even I tried to change the Advance setting of No.of events to 1 in the pubsub connector but still the events are coming with square brackets. Kindly help me how i can remove those square brackets. Thanks,Murali

281

6 months ago

S

sriram9780New Participant

Community Q&A

Can 2 WHEN case statements distribute events to same stream via Router ?

I am trying to modify an existing Striim App which routes events based on the table name to different Target Writers. I would like to know if two different WHEN statements can be routed to the same stream. Based on the Striim Router documentation here . Can the sample code be modified to route the events of two case statements to `stream_one` as shown below. CREATE ROUTER myRouter INPUT FROM mySourceStream AS src CASE WHEN TO_INT(src.data[1]) < 150 THEN ROUTE TO stream_one, WHEN TO_INT(src.data[1]) >= 150 THEN ROUTE TO stream_two, WHEN meta(src,"TableName").toString() like 'QATEST.TABLE_%' THEN ROUTE TO stream_one,ELSE ROUTE TO stream_else;Exploring other possible ways to resolve the above issue - Is it possible to write the above as a Boolean logic? CREATE OR REPLACE ROUTER myRouter INPUT FROM mySourceStream AS srcCASE WHEN meta(src,"TableName").toString() like 'QATEST.USERS' THEN ROUTE TO stream_one OR WHEN meta(src,"TableName").toString() like 'QATEST.ACCOUNTS' THEN ROUTE

210

S

10 months ago

EmrahParticipating Frequently

Community Q&A

CDC problem with 21C

Dear All,i am trying to create 2 pipelines from Oracle to Postgres.the first one is initial Oracle to Postgres, it worked very well, and schema conversion tool migrated 2 example tables to postgres. but i have an issue with Oracle CDC.i have logminer user on oracle which is striim and i have table owner which is scale_user and i have 2 tables scale_data and scale_data2.i am not using container and PDB on 21C, striim and scale_user are common user.i am receieving an error like below when i insert a data to the tables after commit.i see that reader is looking for tables like CDB$ROOT.TEST.SCALE_DATA but i am not using container so how can resolve the issue.There was no problem with initial load, the problem is related with CD process. Thanks in advance. [{ "_id" : null, "timeStamp" : 1707166962232, "originTimeStamp" : 1707166964000, "key" : null, "sourceUUID" : { "uuidstring" : "01eec465-3101-b301-b854-42010a9c000a" }, "data" : [ "862412", "1", "1", "460" ], "metadata" : { "RbaSqn" : "50

2188

1 year ago

B

bobbyilievNew Participant

Product Q&A

MySQL Reader + KafkaWriter: How to specify individual topics for the individual MySQL tables?

Hey! I have the following setup:MySQL → Striim → KafkaThis all works as expected, but if I have multiple MySQL tables, all changes are streamed into 1 single Kafka topic.Is there a way to configure the KafkaWriter to stream the messages to dedicated topics for each table?For exmaple:Events from table `users` should go to a topic `users` Events from table `items` should go to a topic `itmes`As far as I can see in the docs, you can only specify one topic per KafkaWriter.

bobbyilievNew Participant

Product Q&A

Suitable Avro type not found for field error when using MySQL -> Striim -> Kafka

I’ve got the following setup:MySQL → Striim → Kafka + CSR and avro encoding The table that I’m replicating is pretty simple: items | CREATE TABLE `items` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `name` varchar(100) DEFAULT NULL, `category` varchar(100) DEFAULT NULL, `price` decimal(7,2) DEFAULT NULL, `inventory` int DEFAULT NULL, `inventory_updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP, `created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP, `updated_at` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`id`), UNIQUE KEY `id` (`id`)) ENGINE=InnoDB AUTO_INCREMENT=2001 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci However the bigint seems to be causing the following error when Striim starts ingesting messages:Message: KafkaWriter is unable to produce events to shop. Component Name: Kafka_2_1_0_mysql_kafka_csr_Target. Component Type: TARGET. Cause: KafkaWriter is unable to produce events to shop. Cause: Suitable Avro type not found fo

642

B

1 year ago

mahesh_guptaNew Participant

Product Q&A

Is Postgresql to Oracle CDC pipeline possible ?

Hi Team, Can we create a Postgresql to Oracle CDC pipeline to make sure oracle is sync with postgresql post migration. Regards, Mahesh

851

1 year ago

sumekKnown Participant

Community Q&A

Parallel Threads causes out of order events during CDC [SOLVED]

we’ve observed a number of lost updates when running CDC. The state of initial load seems fine, but later on we came across NO_OP_UPDATES. I’ve ignored the exception for the CDC, but that resulted in some of the updates being lost and records left in stale state.It seems that it might be connected with updates being in the same batch as the initial insert. How are events ordered within a single batch? Have you observed such scenarios? Source is SQL Server and target is Postgres in case thats relevant.

1125

1 year ago

S

sriram9780New Participant

Product Q&A

How to retrieve last checkpoint after Source Component has been updated?

For a CDC app using MySQL reader, I use the last checkpoint of the source component using Mon <namespace>.<application name> before I restart an app.After making a change to the source component , the checkpoint info is gone.Is there any other way I can retrieve the last checkpoint?Any tips are deeply appreciated and also curious to see different options available for these kind of scenarios. Thanks in advance!

60

S

20 days ago

emiliogabriel000New Participant

Product Q&A

MySQL data migration to alloydb

Hello, I am participating in a project at the company I work for where we are considering migrating data from a MYSQL database to ALLOYDB but so far I have not found any effective documentation that could help me... could you give me some support with this?

abgNew Participant

Error "Please upload a local file" when uploading MySQL driver in Developer Edition

I receive the Error "Please upload a local file" when I try to upload MySQL driver in the Developer Edition.I have tried to upload MySQL and MariaDB drivers and I always receive the same error.

404

4 months ago

muraliNew Participant

Community Q&A

Removing Square brackets from JSON message before publishing to Pubsub

We are publishing CDC changes from Oracle table to GBQ PubSub, however the Striim is publishing JSON Array with square brackets with one CDC change. I don’t want those flower brackets , how can we resolve this. Even I tried to change the Advance setting of No.of events to 1 in the pubsub connector but still the events are coming with square brackets. Kindly help me how i can remove those square brackets. Thanks,Murali

281

6 months ago

S

sriram9780New Participant

Community Q&A

Can 2 WHEN case statements distribute events to same stream via Router ?

I am trying to modify an existing Striim App which routes events based on the table name to different Target Writers. I would like to know if two different WHEN statements can be routed to the same stream. Based on the Striim Router documentation here . Can the sample code be modified to route the events of two case statements to `stream_one` as shown below. CREATE ROUTER myRouter INPUT FROM mySourceStream AS src CASE WHEN TO_INT(src.data[1]) < 150 THEN ROUTE TO stream_one, WHEN TO_INT(src.data[1]) >= 150 THEN ROUTE TO stream_two, WHEN meta(src,"TableName").toString() like 'QATEST.TABLE_%' THEN ROUTE TO stream_one,ELSE ROUTE TO stream_else;Exploring other possible ways to resolve the above issue - Is it possible to write the above as a Boolean logic? CREATE OR REPLACE ROUTER myRouter INPUT FROM mySourceStream AS srcCASE WHEN meta(src,"TableName").toString() like 'QATEST.USERS' THEN ROUTE TO stream_one OR WHEN meta(src,"TableName").toString() like 'QATEST.ACCOUNTS' THEN ROUTE

bobbyilievNew Participant

Product Q&A

MySQL Reader + KafkaWriter: How to specify individual topics for the individual MySQL tables?

Hey! I have the following setup:MySQL → Striim → KafkaThis all works as expected, but if I have multiple MySQL tables, all changes are streamed into 1 single Kafka topic.Is there a way to configure the KafkaWriter to stream the messages to dedicated topics for each table?For exmaple:Events from table `users` should go to a topic `users` Events from table `items` should go to a topic `itmes`As far as I can see in the docs, you can only specify one topic per KafkaWriter.

bobbyilievNew Participant

Product Q&A

Suitable Avro type not found for field error when using MySQL -> Striim -> Kafka

I’ve got the following setup:MySQL → Striim → Kafka + CSR and avro encoding The table that I’m replicating is pretty simple: items | CREATE TABLE `items` ( `id` bigint unsigned NOT NULL AUTO_INCREMENT, `name` varchar(100) DEFAULT NULL, `category` varchar(100) DEFAULT NULL, `price` decimal(7,2) DEFAULT NULL, `inventory` int DEFAULT NULL, `inventory_updated_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP, `created_at` timestamp NULL DEFAULT CURRENT_TIMESTAMP, `updated_at` datetime DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (`id`), UNIQUE KEY `id` (`id`)) ENGINE=InnoDB AUTO_INCREMENT=2001 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci However the bigint seems to be causing the following error when Striim starts ingesting messages:Message: KafkaWriter is unable to produce events to shop. Component Name: Kafka_2_1_0_mysql_kafka_csr_Target. Component Type: TARGET. Cause: KafkaWriter is unable to produce events to shop. Cause: Suitable Avro type not found fo

642

B

1 year ago

sumekKnown Participant

Community Q&A

adjusting timezone

I’d like to convert a timestamp from location specific one (in areas that have summer time change) to UTC. What would be the best way to do it within Striim? I assume it’s doable using Joda API and I can see in the reference that “Striim supports all date functions natively associated with Joda-Time.” but I didn’t find an example how to call those Joda Time functions

1355

1 year ago

sploiselleNew Participant

Product Q&A

MySQL to Kafka+CSR

Trying to set up:Source: MySQL replication hosted in RDS, which works on its ownTarget: Kafka (works on its own) + CSR (untested) hosted on Confluent Cloud--using the AvroFormatter.The CSR is untested bc it doesn't seem like CSR is supported w/ the builtin load generators--though maybe I'm wrong.When I deploy it, I get:Error Deploy failed! java.util.concurrent.ExecutionException: java.lang.reflect.InvocationTargetExceptionI’ve tested the connection to the CSR and it works, so I don’t believe it’s a credential issue.

772

1 year ago

S

sriram.pasamCommunity Manager

Product Q&A

Syntax error in Continuous query(CQ) to inject NULL value for String field value

731

S

1 year ago

Need More Help?

Contact Support

Reach out to our world class support team

Talk To Us

Need additional support? We have a network of certified partners eager to help!

Striim |

Legal |

Privacy Policy

Powered by Gainsight

Terms & Conditions Cookie settings

Sign up

Already have an account? Login

Login with SSO

or

Username *

E-mail address *

Password *

I accept the terms & conditions

loginBox.register.email_repeat

Login to the community

No account yet? Create an account

Login with SSO

or

Username or Email

Password

Remember me

Forgot password?

Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.

Enter your e-mail address

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

OK

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.

OK