In Google Cloud Platform Cloud SQL, the following error log is generated every second. What are the possible causes?
Google App Engine has stopped, so I don't think it is the cause.
Aborted connection 1000 to db: ‘unconnected’ user: ‘user-name’ host: ‘localhost’ (Got an error reading communication packets)`
Related
I am trying to setup a mssql debezium connector with AWS MSK Connect but keep getting the following error messages:
Connector error log:
[Worker-0a949760f6b805d4f] [2023-02-15 19:57:56,122] WARN [src-connector-014|task-0] [Consumer clientId=dlp.compcare.ccdemo-schemahistory, groupId=dlp.compcare.ccdemo-schemahistory] Bootstrap broker b-3.stuff.morestuff.c7.kafka.us-east-1.amazonaws.com:9098 (id: -2 rack: null) disconnected (org.apache.kafka.clients.NetworkClient:1079)
This error happens continuously for a bit then I see this error:
org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata
In the cluster logs I see a corresponding error when I get the disconnect error:
[2023-02-15 20:08:21,627] INFO [SocketServer listenerType=ZK_BROKER, nodeId=3] Failed authentication with /172.32.34.126 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
I have an ec2 client that i've setup to connect to my cluster and am able to connect and run commands against the cluster using IAM auth. I have setup a topic and produced and consumed from the topic using the console producer/consumers. I've also verified that when the connector start up it is creating the __amazon_msk_connect_status_* and __amazon_msk_connect_offsets_* topics.
I've verified that ip in the logs is the ip assigned to my connector by checking the Elastic Network Interface it was attached to.
Also for testing purposes I've opened up all traffic from 0.0.0.0/0 for the SG they are running in and also made sure the IAM role has msk*, msk-connect*, kafka*, and s3*.
I've also verified CDC is enabled on the RDS and that it is working properly. I see changes being picked and added to the CDC tables.
I believe the issue is related to IAM auth still but am not certain.
Cluster Config:
auto.create.topics.enable=true
delete.topic.enable=true
worker config:
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
config.providers.secretManager.class=com.github.jcustenborder.kafka.config.aws.SecretsManagerConfigProvider
config.providers=secretManager
config.providers.secretManager.param.aws.region=us-east-1
request.timeout.ms=90000
errors.log.enable=true
errors.log.include.messages=true
Connector Config:
connector.class=io.debezium.connector.sqlserver.SqlServerConnector
tasks.max=1
database.history.consumer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
schema.include.list=dbo
database.history.producer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
database.history.consumer.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
database.history.consumer.security.protocol=SASL_SSL
database.instance=MSSQLSERVER
topic.prefix=dlp.compcare.ccdemo
schema.history.internal.kafka.topic=dlp.compcare.ccdemo.history
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter=org.apache.kafka.connect.storage.StringConverter
database.history.sasl.mechanism=AWS_MSK_IAM
database.encrypt=false
database.history.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
database.history.producer.sasl.mechanism=AWS_MSK_IAM
database.history.producer.sasl.jaas.config=software.amazon.msk.auth.iam.IAMLoginModule required;
database.user=debezium
database.names=Intermodal_CCDEMO
database.history.producer.security.protocol=SASL_SSL
database.server.name=ccdemo_1
schema.history.internal.kafka.bootstrap.servers=b-1:9098
database.port=1433
database.hostname=my-mssql-rds.rds.amazonaws.com
database.history.sasl.client.callback.handler.class=software.amazon.msk.auth.iam.IAMClientCallbackHandler
database.password=${secretManager:dlp-compcare:dbpassword}
table.include.list=dbo.EquipmentSetup
database.history.security.protocol=SASL_SSL
database.history.consumer.sasl.mechanism=AWS_MSK_IAM
I was able to do this same process but with a postgres rds with no issues.
I've tried everything I can think of so any an all help would be greatly appreciated!
I also referenced the following when setting up the cluster/connector:
https://catalog.workshops.aws/msk-labs/en-US/mskconnect/source-connector-setup
https://thedataguy.in/debezium-with-aws-msk-iam-authentication/
https://debezium.io/documentation/reference/stable/connectors/sqlserver.html#sqlserver-connector-properties
Streaming MSSQL CDC to AWS MSK with Debezium
https://docs.aws.amazon.com/msk/latest/developerguide/mkc-debeziumsource-connector-example.html
I have made a few pipelines in Azure Data Factory, which transfer and modify data from Blob Storage (Excel Files) to Azure SQL. They were off for like 2 month and the company has implemented MFA on whole Azure Active Directory.
After that when I try to run the pipelines I have only "Failed status". For every pipeline the error is the same. They are look like this:
Operation on target Data flow1 failed: {"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: java.lang.Exception: fail to reach https://we.frontend.clouddatahub.net/subscriptions/aa2d32bf-f0d0-4656-807b-7e929da73853/entities/99264214-3071-4faa-87c2-32d9dec7e5a4/identities/00000000-0000-0000-0000-000000000000/token?api-version=2.0 with status code:403, payload:{"error":{"code":"ManagedIdentityInvalidCredential","message":"Acquire MI token from AAD failed. ErrorCode: invalid_client, Message: A configuration issue is preventing authentication - check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS700027: Client assertion failed signature validation.\r\nTrace ID: 4eef805e-a0ca-494e-bcc2-c01cd755f400\r\nCorrelation ID: f313ba30-9455-4065-90ab-a0fe28dadc99\r\nTimestamp: 2022-02-21 13:11:56Z","details":[],"additionalInfo":[]}}, CorrelationId:171b73ff-5721-45e5-bf95-2b29dc4dd1b4, RunId:887b22ec-6cae-42d3-9580-b93a98800b3c","Details":"java.lang.Exception: fail to reach https://we.frontend.clouddatahub.net/subscriptions/aa2d32bf-f0d0-4656-807b-7e929da73853/entities/99264214-3071-4faa-87c2-32d9dec7e5a4/identities/00000000-0000-0000-0000-000000000000/token?api-version=2.0 with status code:403, payload:{"error":{"code":"ManagedIdentityInvalidCredential","message":"Acquire MI token from AAD failed. ErrorCode: invalid_client, Message: A configuration issue is preventing authentication - check the error message from the server for details. You can modify the configuration in the application registration portal. See https://aka.ms/msal-net-invalid-client for details. Original exception: AADSTS700027: Client assertion failed signature validation.\r\nTrace ID: 4eef805e-a0ca-494e-bcc2-c01cd755f400\r\nCorrelation ID: f313ba30-9455-4065-90ab-a0fe28dadc99\r\nTimestamp: 2022-02-21 13:11:56Z","details":[],"additionalInfo":[]}}, CorrelationId:171b73ff-5721-45e5-bf95-2b29dc4dd1b4, RunId:887b22ec-6cae-42d3-9580-b93a98800b3c\n\tat com.microsoft.datafactory.dat"}
Is there any way I can evade this error without deactivating MFA?
Thank you David Browne - Microsoft for your valuable suggestion. Posting your suggestion as answer to help other community members.
Use either of Managed identity or Provision a Service principle
for authentication. Switch the Authentication to SQL Auth for SQL Server and SAS/Account Key auth for Azure Storage.
I’m configure matomo server in AWS cloud with server of 4 nucleos and 16GB RAM. The database is in RDS da AWS. But when I go to all websites and select data renge, I get the folloing error message: “Oops… there was a problem during the request.” I set all recommended settings. The other options in AllWebsites return the reports.
It sounds like there is an error on the Matomo servers and that the PHP process is failing with an error.
-> Can you please get the team to check your Web server error.log for the actual error message(s) logged by the server?
Once we see the error messages we will likely be able to understand where the problem is, and help provide a solution.
We have a website running on a server. We have a "production" instance and a "staging" instance each having its own database. The MSSQL Server is running locally on the same server.
Today, suddenly the "production" website went down. Looking at the logs, the following exception showed up:
System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
At the same time, the "staging" website was working just normally.
While trying to figure out what was happening, I tried all sorts of things like re-creating both the app pool and the IIS app. I also hooked up the "production" IIS app to the same app pool of the "staging" app, still the same issue. Restarted the server too of course.
Also, I ran the executable of the "production" website directly (as a console app) and it worked normally. So it's a problem that happens only when running under IIS.
One last thing I tried, is that I reconfigured the "staging" website to use the "production" database, and to my utter shock it worked normally. Because I thought the problem was the "production" database itself.
I just have no idea whatsoever about what's going on here. Any help is very much appreciated.
If all the connections in the connection pool are used, it is almost certainly because your application is opening database connections and failing to close them.
Since you are using Entity Framework, it's probably because your application is failing to dispose of the DbContext object.
It's nothing to do with the production database as such; probably the increased activity on your production site vs your staging site is making the application bug manifest itself more quickly.
I followed the example on this website to implement a GCM server using CCS. However the code has exception when it tries to connect to the gcm server (last line in the code below):
ConnectionConfiguration config = new ConnectionConfiguration(GCM_SERVER, GCM_PORT);
config.setSecurityMode(SecurityMode.enabled);
config.setReconnectionAllowed(true);
config.setRosterLoadedAtLogin(false);
config.setSendPresence(false);
config.setSocketFactory(SSLSocketFactory.getDefault());
connection = new XMPPTCPConnection(config);
connection.connect();
I looked up online and someone said I needed to enable billing for my app on appengine in order to use GCM server. I did so but it still does not work. I keep seeing the following error:
gcm.googleapis.com:5235 Exception: Permission denied: Attempt to
access a blocked recipient without permission. (mapped-IPv4)
Am I missing something?