I have developed an email client at work using JavaMail. Some customers have reported massive delays when searching for subjects with non-English characters (e.g. "Kühlgut").
It turned out that some mail providers (in this case Office 365) do not support searching with UTF-8 charset (resulting in a BadCommandException). Therefore, JavaMail fetches and performs the search locally (as fallback).
This behaviour caused the massive delays.
Code:
String searchTerm = "Kühlgut";
SearchTerm[] searchTerms = new SearchTerm[3];
searchTerms[0] = new FromStringTerm(searchTerm);
searchTerms[1] = new SubjectTerm(searchTerm);
searchTerms[2] = new BodyTerm(searchTerm);
OrTerm orTerm = new OrTerm(searchTerms);
folder.search(orTerm);
JavaMail debug output of the search command:
C18 SEARCH CHARSET UTF-8 OR OR FROM {8+}
Kühlgut SUBJECT {8+}
Kühlgut BODY {8+}
Kühlgut ALL
C18 NO [BADCHARSET (US-ASCII)] The specified charset is not supported.
C19 SEARCH CHARSET Cp1252 OR OR FROM {7+}
Kühlgut SUBJECT {7+}
Kühlgut BODY {7+}
Kühlgut ALL
C19 NO [BADCHARSET (US-ASCII)] The specified charset is not supported.
B20 FETCH 1 (ENVELOPE INTERNALDATE RFC822.SIZE)
* 1 FETCH (ENVELOPE ....)
B20 OK FETCH completed.
B21 FETCH 1 (BODYSTRUCTURE)
* 1 FETCH (BODYSTRUCTURE ...
B21 OK FETCH completed.
I do have two questions:
Is it possible to check if the IMAP server supports searching with UTF-8 charset (e.g. via capabilities)?
How can I stop JavaMail from fetching all emails if the former is not possible (no exception is thrown)?
I don't think there's a way to do #1, but for #2 try setting the mail.imap.throwsearchexception property to true.
And submit a bug to Microsoft since Office 365 really should support searching with utf-8.
Related
We have a requirement to set up Kafka MicrosoftSqlServerSource connect.
This is to capture all the transactions(insert/update) performing in one of the sales table in Azure SQL database.
In order to bring the support for the above source connect, we have initially enabled CDC at both database and table level.
We also created a view out of the source table which will be the input for the source connect( TableType = VIEW in connector configuration).
Once we complete the set up at both connector as well as database level, we could see messages flowing to the respective topic created automatically along with the connector as when a new updations/insertions happened at the table level.
One strange behavior we observed while testing is that when we stopped the testing, the last message received in the topic starts getting duplicated until a new message arrived.
Could you please help us to understand whether this is a system behavior?
Or Did we miss any configuration that has resulted in these duplicate entries.
Please guide us on how we can tackle the above duplicate issue.
Attaching the snapshot
Connector Summary
Connector Class = MicrosoftSqlServerSource
Max Tasks = 1
kafka.auth.mode = SERVICE_ACCOUNT
kafka.service.account.id = **********
topic.prefix = ***********
connection.host = **************8
connection.port = 1433
connection.user = ***************
db.name = **************88
table.whitelist = item_status_view
timestamp.column.name = ProcessedDateTime
incrementing.column.name = SalesandRefundItemStatusID
table.types = VIEW
schema.pattern = dbo
db.timezone = Europe/London
mode = timestamp+incrementing
timestamp.initial = -1
poll.interval.ms = 10000
batch.max.rows = 1000
timestamp.delay.interval.ms = 30000
output.data.format = JSON
What you're describing is controlled by
mode = timestamp+incrementing
poll.interval.ms = 10000
It should save the last timestamp, then query only for timestamps greater than the last... If you are getting greater than or equal to, then that is certainly a bug that should be reported.
Or you should read the docs
A timestamp column must use datetime2 and not datetime. If the timestamp column uses datetime, the topic may receive numerous duplicates
As an alternative, you could use Debezium (run your own Connector, not use Confluent Cloud offering) to truly stream all table operations.
I am trying to get some user attributes from Active Directry using Windows LDAP API. I am using:
Active Directory Version: The one that comes with Windows Server 2012
LDAP version: 3
Wldap32.lib version: The one that comes with Windows 10 x64
Eg:
PCHAR myAttributes[4];
myAttributes[0] = "DistinguishedName";
myAttributes[1] = "DisplayName";
myAttributes[2] = "PasswordExpired";
myAttributes[3] = "mail";
ldap_search_s(
myLdapConnection, // Ldap connection
myDomain, // DN to start search
LDAP_SCOPE_SUBTREE, // Scope
myFilter, // Filter
myAttributes, // Retrieve list of attributes
0, // Get both attributes and values
&mySearchResult // [out] Search results
);
It returns DistinguishedName, DisplayName and mail attributes, but does not reuturn the PasswordExpired attribute.
I queried with some other attributes and it looks like it does not return attributes with boolean values as well as the EmailAddress attribute.
Why does it not return PasswordExpired attribute?
What about EmailAddress ?
Is there a difference between EmailAddress and mail ?
There is no PasswordExpired LDAP attribute in Active Directory. To build your query look at this URL for attribute names in standard Active Directory schema https://msdn.microsoft.com/en-us/library/ms675090(v=vs.85).aspx. To check if password for given account is expired you'll need to check userAccountControl attribute, which is actually value storing flags of different states of user account https://msdn.microsoft.com/en-us/library/ms680832(v=vs.85).aspx. There is IADsUser interface that will translate this all for you in case you don't have to stick just to LDAP https://msdn.microsoft.com/en-us/library/aa746343(v=vs.85).aspx
You will find current primary email address in mail attribute. There is no EmailAddress LDAP attribute, unless you meant E-mail-Addresses which is CN for the same schema attribute as mail, so no difference there.
See above. In general if you do not have compelling reason to stick just to C/LDAP I'd recommend you to use .Net Framework instead. Otherwise you have a lot of work ahead of you - not just interpreting bit flags like in case of password expiration but possibly also with different authentication methods, different structures capturing time and date, accounting for timezones, UTF, chasing referrals and other stuff you might need depending on complexity of what you want to achieve. You will be productive much faster in .Net Framework. See DirectoryServices https://msdn.microsoft.com/en-us/library/mt481534(v=vs.110).aspx namespace or Security namespace https://msdn.microsoft.com/en-us/library/mt481561(v=vs.110).aspx for details.
In order to send automated job failure notifications from SQL Server Agent, you must configure Database Mail and then to to SQL Agent properties > Alert System > Mail Session > Enable mail profile to configure the mail system and mail profile to be used to send the email notifications. We have many servers, and would like to setup a central cross-server job that ensures that the Enable mail profile option is checked across various servers, because otherwise, the scheduled jobs fail silently without sending an email notification.
Is there a supported way to query the msdb database to get to these settings using T-SQL (or by some other way programmatically)?
Running a SQL Profiler trace while bringing up the properties page in the SQL Server Agent UI shows references to msdb.dbo.sp_get_sqlagent_properties which is an undocumented procedure (I would prefer to use documented objects to help future-proof our solution), and several calls to master.dbo.xp_instance_regread which I would imagine the reg keys could change per each SQL Server instance installation.
Does anyone know of a way to query to check whether the enable mail profile option is configured, and also retrieve the mail profile that is designated in the SQL Agent Alert System configs? Most of our servers are SQL Server 2008 R2, with some SQL 2012. I would prefer SQL 2008+ support.
Thanks in advance!
Future-proofing is a very wise idea :). As you noted, both xp_regread and xp_instance_regread are also undocumented. And http://social.msdn.microsoft.com/Forums/sqlserver/en-US/b83dd2c1-afde-4342-835f-c1debd73d9ba/xpregread explains your concern (plus, it offers you an alternative).
Your trace and your run of sp_helptext 'sp_get_sqlagent_properties' are a good start. The next thing to do is run sp_helptext 'sp_helptext', and note its reference to sys.syscomments. BOL sys.syscomments topic redirects you to sys.sql_modules, and that points to the next step. Unfortunately for your needs, just one row (for 'sp_get_sqlagent_properties') will be returned by running USE msdb; SELECT object_name(object_id) FROM sys.sql_modules WHERE definition LIKE '%sp_get_sqlagent_properties%'. I thus assume you are out of luck - there appears to be no alternative, publicly documented, module (sproc). My assumption could be wrong :).
I deduce that xp_reg% calls exist for client (SMO, SSMS, etc.) needs, such as setting/getting agent properties. More importantly (for your needs), your sp_helptext run also reveals SSMS (a client) is using a registry store (i.e. not a SQL store). Unfortunately, I must deduce (based upon an absence of proof from a library search) that those keys (and their values) are also not documented...
The above appears to put you in a pickle. You could decide "if we are going to rely upon undocumented registry keys, we might as well rely on the undocumented calls to read them", but I won't recommend that:). You could also file a feature request at https://connect.microsoft.com/ (your need is clear), but because your need concerns a client-side feature request, I do not recommend holding your breath while waiting for a fix :).
Perhaps it is time to step back and take a look at the bigger picture:
How often can that key be changed, and how often will this process poll for that change?
Email uses a mail primitive. Sender: "Dear recipient, did you get my mail?" Recipient: "Dear sender, did you send me mail?" Disabling an email profile is not the only reason for an email failure.
Would a different approach be more useful, when compared to periodically checking a key?
One approach would be to periodically send "KeepAlive" email. If the "KeepAlive" email isn't periodically received, maybe that key was tweaked, maybe the Post Office is on a holiday, or maybe something else (equally bad) happened. Plus, this approach should be fully supported, documented, and be future-proof. Who knows how and what keys will be used in the next version of SQL Server Agent?
The first bullet isn't addressed (neither would it be addressed by periodically checking a key), and perhaps you have additional needs (worth mentioning on MS connect).
I finally found a way to do this using PowerShell and Microsoft.SqlServer.Management.Smo.Agent.JobServer.
Here is the PowerShell script I wrote to check if SQL Agent mail alerts are enabled, and to make sure that SQL Agent is set to Auto startup when the server reboots. This works works with local or remote SQL instances.
# usage examples
Check-SQLAgentConfiguration -InstanceName 'localhost\sql2014'
Check-SQLAgentConfiguration -InstanceName 'RemoteServerName'
function Check-SQLAgentConfiguration{
param([string]$InstanceName='localhost')
[System.Reflection.Assembly]::LoadWithPartialName('Microsoft.SqlServer.SMO') | out-null;
$smosrv = new-object Microsoft.SqlServer.Management.Smo.Server($InstanceName);
$smosrv.ConnectionContext.ConnectTimeout = 5; #5 seconds timeout.
$smosrv.ConnectionContext.ApplicationName = 'PowerShell Check-SQLAgentConfiguration';
"Server: {0}, Instance: {1}, Version: {2}, Product Level: {3}" -f $smosrv.Name, $smosrv.InstanceName, $smosrv.Version, $smosrv.ProductLevel;
# NOTE: this does not seem to ever fail even if SQL Server is offline.
if(!$smosrv){"SQL Server Connection failed"; return $null;}
$agent = $smosrv.JobServer;
if(!$agent -or $agent -eq $null){
throw "Agent Connection failed";
return -2;
}
$agentConfigErrMsg = "";
if($agent.AgentMailType -ne "DatabaseMail"){ $agentConfigErrMsg += " AgentMailType: " + $agent.AgentMailType + "; "; }
if(!$agent.DatabaseMailProfile){$agentConfigErrMsg += " DatabaseMailProfile: " + $agent.DatabaseMailProfile + "; ";}
if($agent.SqlAgentAutoStart -ne "True"){$agentConfigErrMsg += " SqlAgentAutoStart: " + $agent.SqlAgentAutoStart + " ServiceStartMode: " + $agent.ServiceStartMode + "; "; }
if($agentConfigErrMsg.length -gt 0){
$agentConfigErrMsg = "Invalid SQL Agent config! " + $agentConfigErrMsg;
throw $agentConfigErrMsg;
return -1;
}
<##
#for debugging:
"Valid: "
"AgentMailType:" + $agent.AgentMailType;
"DatabaseMailProfile: " + $agent.DatabaseMailProfile;
"ServiceStartMode: " + $agent.ServiceStartMode;
"SqlAgentAutoStart: " + $agent.SqlAgentAutoStart;
#"SqlAgentMailProfile: " + $agent.SqlAgentMailProfile;
#>
return 0;
}
SQL 2008R2 uses Service-broker like queues for mail processing (http://technet.microsoft.com/en-us/library/ms175887%28v=sql.105%29.aspx). In our environments I check that the corresponding queue exists and is active.
SELECT * FROM msdb.sys.service_queues
WHERE name = N'ExternalMailQueue'
AND is_receive_enabled = 1;
This table is listed online (http://technet.microsoft.com/en-us/library/ms187795%28v=sql.105%29.aspx).
Testing shows that this makes the required transition as we went from new instance -> enabled mail -> mail switched off and back again.
I am using Javamail library to fetch emails from several servers through IMAP.
I only care for the unread messages and I want to download only the 5 last received unread messages.
For filtering the messages in a folder I am using using the Folder.search(FlagTerm ft) method passing the flag SEEN with value false, just as following code shows:
FlagTerm ft = new FlagTerm(new Flags(Flags.Flag.SEEN), false);
Message[] messages = folder.search(ft);
I need to diminish bandwidth usage and the above method may return an arbitrarily large number of messages. I am only interested in the last 5 of them, is there a way for the IMAP server return a limited number of messages?
You can do the search over a subset of the messages, effectively setting an upper limit on the number of messages returned, but you might have to do multiple searches. There's no direct way to limit the number of results returned if you're searching all messages.
Note that the search results are relatively compact (effectively only the message number), so unless you're searching a huge number of messages I wouldn't think bandwidth would be an issue relative to fetching the content of the messages.
Performance Engineering tool: LoadRunner 11
Protocol: Silverlight
The scenario:
1. A call is made (by the Silverlight application) to the web server to generate a unique identifier (UID)
2. The server returns a response with the UID
3. The application uses that UID to save Patient record (in this scenario, UID is Unique Patient Id)
The problem:
We would like to use the ID received in the Step 2's Web Response (say, as a local variable) and replace it in the next subsequent request sent by LoadRunner.
Please advise whether the same is possible.
*In parallel trying to figure out if "web_reg_save_param" will solve our problem.*
------------ LoadRunner data --------------------
1. Call to generate Uid
ignored as it is too large to paste here and irrelevant
2. Response from server (The UID is UNI-0000001544)
HTTP/1.1 200 OKCache-Control: privateContent-Type: application/msbin1Content-Encoding: gzipVary: Accept-EncodingServer:
Microsoft-IIS/7.5X-AspNet-Version: 4.0.30319X-Powered-By: ASP.NETDate:
Fri, 06 Jul 2012 05:41:27 GMTContent-Length:
188#GenerateSequenceResponsehttp://tempuri.org/#GenerateSequenceResult™UNI-0000001544
3. Next LoadRunner request where the UID (UNI-0000001543) has to be replaced with the UID received in the response
web_custom_request("SubmitChanges",
"URL=http://infinityappload/ClientBin/Infinity-Web-Services-ActorDomainService.svc/binary/SubmitChanges",
. . .
, "BodyBinary=#\rSubmitChanges\
. . .
\tPatientID\\x99\\x0EUNI-0000001543\
... LAST);
This is, as James said, the idea of correlation. The simplified approach would go as follows:
web_reg_save_param("UID2", "LB=GenerateSequenceResult™", "Savelen=14", LAST);
//The TM symbol will have to be replaced by whatever HTML code is used.
web_custom_request("WebRequest1", ....);
web_custom_request("SubmitChanges", "URL=http://infinityappload/...",
...
"BodyBinary=#\rSubmitChanges\",
"\tPatientID\\x99\\x0E{UID2}\",
LAST);
Values that are to be correlated are going to be determined prior to the request they will be populated by. They can then be used in any subsequent requests. Just make sure that, when using a correlated parameter, it is inside a quoted string.
The concept you are referring to is the management of correlated variables, it is a core concept in the use of performance test tools, LoadRunner included. This particular concept is the subject of nearly 1/3 of the standard LoadRunner script development product training. I would refer you to information on correlation for both web and silverlight protocols in the system documentation. Also, it is critical for your short and long term success that you pair yourself with both a strong mentor as well as attending some form of standard product training.