SQL where condition automation using python - sql-server

I have a python based client and server application. Client sends some data to the server based
on which server (running on python) runs some query on SQL SERVER database. For example let's consider client sends
a=5,b=10
Based on which Server computes :
#Server.py
import pyodbc
def test(a,b):
cnxn = pyodbc.connect(r'Driver={SQL Server};Server=.;Database=testdb;Trusted_Connection=yes;')
cursor = cnxn.cursor()
cursor.execute("select * from sample where a='"+a+"'" and b='"+b+"'"")
for row in cursor:
#doSomething ()
Both the variabes a and b are hardcoded inside the execute function.
Now what if I want to automate the whole process and by automate I mean to say what if
client sends c,d,e,f etc along with a,b. Do I have to change the server code every time or is there any
process to automate the whole server code irrespective of what client sends?
Thank You

It should have many solutions but depending on your architecture and the protocols between clients and the server.
For example,
We can define the message between clients and the server must follow below rules:
The message must be one json
The message must has one node named 'MessageType' or 'MessageID' or 'MessageName'
For your scenario, the message will be like
{'MessageType':'BeginSchedule', 'a':'test1', 'b':'test2', 'option1':1, 'option2':'t1', 'ClientID':'xxx'}
When the server receives one message from one client and its message type is 'BeginSchedule', then will invoke 'test(a,b)' to consume this request.
If there is another schedule, we can define one 'MessageType'='BeginSchedule2', then implement one handler in server code to consume any request whose MessageType='BeginSchedule2'.
The clients may send dirty data inside the message, you have to add some validators before calling the handler.
The clients may send extra data inside the message, you can design them as the optional arguments for the handler, please refer to:
Using Optional and Named Arguments
If you need your handlers accept infinite arguments, please refer to:
How to use *args and **kwargs in Python

Related

How to ingest error logs from processors to database table

I want to add all my failure logs to a column in DB.
As I am new to nigi. I'm confused on how to do that.
Need suggestion on how to do that.
You can follow the steps outlined below.
Under Controller Settings, add a REPORTING TASKS of type SiteToSiteBulletinReportingTask. This will capture bulletins. Make sure that Input Port Name property value is exact same as input port created at step 2. Configure rest of properties accordingly.
Add an input port at root level of NiFi i.e. not inside any process group.
When you start the reporting task, it will start sending bulletins to configured input port, i.e. NiFi instance will send data to itself, and you can further ingest into database.
Have a look at these implementation resources,
https://medium.com/#ashish211981/nifi-monitoring-driven-development-6b21046ee9e4
https://www.clearpeaks.com/health-monitoring-of-nifi-flows-using-bulletin-events/

Duplex communication between Client and server open62541(OPC-UA)

I would like to establish bidirectional data communication between client and server. The following example explains what I am trying to acheive.
Ex: Client sends a request to read the value of the two node ids(these node ids are numeric) every sec. Then in the client program it adds these numbers and gives it to the user.
Then after 60 seconds ,the server tells the client to multiply the number .So now every second the client is multiplying the values of the node ids. Then after 60 seconds , it switches back to addition again.
I am able to request the data from the server through the client but I am trying to figure out how to make server request for data from the client?
Edit: I would like to add some new information to my original question.The reason is to make the post more clearer.The following image describes what I am trying to acheive.
In the above image,provider can be visualised as a server method and consumer can be visualised as a client method. Now if safety consumer2(client2) is connected to safety provider2(server2),how to pass the data received from safety provider2 to safety provider1.(as client can only be connected to a single server).Does the features of open62541 allow the following data flow:
safety provider 2-->safety consumer2 --> safety provider1? If yes, can you provide an example? Thanks
That's not really how OPC UA works.
Instead, your client could monitor the Value attribute of some VariableNode in the server and then behave a certain way based on the current value.
The OPC UA protocol does what its documents specify and what you ask is not possible.
But you can make the client read both the values and the multiplier and do the operations you want on the client side, apart from OPC UA

Is this system an optimal solution to sync an app with a server in real time efficiently?

Problem
I have an Android and iOS app, looking like a classic social network. I need to update UI in real time. Currently, I use a classic system of a client polling each second to a php script by HTTP. The php script bother the database every second for every client and responds, most of the time that there is no new update. If there is a new update, the php script process it and send it back to the client app.
There are 3 problems in this approach : (1) slow user experience (1 second delay each time) + high battery and data usage, (2) apache machines bothered each second by incoming HTTP request, (3) database machine bothered each second by the apaches machines (requesting if they are new stored updates in the main database).
I feel that this system could be substentially improved. For problem (1), I know a TCP connection can be "piped" to the app, but there is still problem (3) because the thread behind the socket still polls the database each second to know if they are new stored updates for their member ID.
Solution ?
I thought of a system to get rid of any activity (client, apaches and database) if there are no new updates. There would be : N apaches server on N machines, a load balancer exposed to the Internet. Behind, these apache server, connected only to local network, 1 "central" database and one "update" database, dedicated for the update system. The "update" database would store 2 tables :
1 table for the mapping between user tokens (and their member ID), and the thread ID and name of current apache machine holding the thread. One user ID may have several connection tokens, but one connection token is associated to only one unique couple (PID - machine name). Each time a user connects to the app, it would create a TCP con held by one thread (in one apache machine), and the [thread ID - machine name] would be stored in that table.
1 table to store the updates themselves. They contain all the informations needed to get up-to-date data (either in raw primitive form like string or int, or in "reference" form, telling the recipient TCP threads it needs to compute "at sending time" some params, for more complex data structures)
The system would be the following :
(1) A user wants to send a message to another user. The app client of the sender sends an HTTP request to the app API endpoint; the load balancer forwards the request to one of the apache machines.
(2) The apache server requests the main database to insert the "user message" row.
(3) The apache server requests the "update" database to know if the recipient has any currently connected device.
(4) if there is at least one connected device, insert an "update" row in the "update" database with all the informations needed, and wake up all thread associated to the recipient user ID (maybe using C signals ?).
(5) All the thread(s) associated to the recipient user ID wake up, they look in the "update" database for new updates associated with their user ID, they process their parameters (especially if there are references params to be computed), they send them back to the recipient devices via TCP.
So my final question is : is such a system feasible, reliable and if so, do you think it can be optimal in term of database and apache machines performence ?
I'm more a front-end programmer and I'm not used to implement complex server architecture, so I wanted to have some opinions before diving into the code, especially if I missed something in my approach (storing PIDs is reliable ? Is it possible for one machine to wake up a thread in another machine through local network ? ...)
PS : I already tried Firebase cloud messaging, but the problem is that they authorize only a one dimension array to be sent with update params. When dealing with complex data structure (like a "user message"), when I receive a signal from FCM in my client app, I still need to make an extra HTTP call to my server to retrieve the new "user message" JSON payload. So, good for my apaches and databases machines (they are not bothered when there is no new updates), bad for the client app that has to send additional HTTP requests. Once again, tell me if I missed something here :)
Thanks for reading

Pushing data across App Engine instances

Let's say we have several clients connected to App Engine using Channel API. Each client sends messages, which should be propagated to other conntected clients according to some rules. The tricky part is that clients may not be to the same App Engine instance.
Is there any way to push data from one instance to the others?
(Yes, I know about Memcache, but this would require some kind of polling.)
You're asking two questions here.
a. Can you push data from one instance to another without the use of polling. The answer is generally no.
b. Can one client send messages to the server that can be propagated to other clients? Yes, and this does not require propagating messages to other server-side instances.
Consider the Channel API as a service. Clients are connected to the Channel API service; they are not connected to any particular instance. Therefore any instance can send messages to any client.
You'll need to store the Channel tokens of your clients in the datastore, in some way that's queryable to match your rules.
Your client makes an HTTP request to send a message to your server.
The handler on the server queries for channel tokens that it needs to propagate the message to (either from memcache or datastore).
The handler on the server sends messages to all the clients.
If the list of destination clients is extremely large, you might want to do steps 3/4 in a task queue where the operation can run longer.
It does not matter what instance a client is connected to, that's hidden from you by the API.
Clients can only "reply" to message via standard HTTP commands, they don't actually have any way to respond via the channel API directly.
So Client A on server A1 wants to sent a message to client B on server B1.
Client A posts to a handler. That might be instance A1 or B1. It does not matter which as the server now passes the message on to client B whatever server client B is connected to via the Channel API.
The real point is that no App Engine instance has any data at all, in general. So it does not matter which instance you connect to, it might be the 99th instance or the very first to start up. So you have to design your application so that it's irrelevant what instance is in use.
Client sends message to server via HTTP.
Server sends message to N clients via the channel API.
Channel API does not make a fixed frontend-instance-to-client connection. Any frontend instance can push message to channel if it knows the channel ID.
What you need to do is pass messages cross-channel.
User one sends message normally to server (e.g. via GET)
Server looks up channel ID of second user and pushes the message
Repeat procedure in other direction: second user to first user.

Designing an interactive client

I'm trying to design a client program that connects to a remote server and sends various messages / request to it and expects responses based on the requests sent (for e.g. send a join message and wait for a response, then either query for some resource or ask for some info etc. in no particular order).
I would like to design the client such that the user can choose any of the possible requests to send after joining the server (after completing one request and getting a response if any it should allow them to carry out further requests or quit). Something like a menu of actions that it returns to each time (while also waiting for any data from the server)? However I can't seem to figure out how to this could be done. Is there a way to do this (preferably without getting into forking/threads)?
Any inputs on this would be really great. TIA
I would start off with a simple chat server to get your feel for socket programming. Google Example TCP Chat Server or something, you'll end up with simple examples like this: http://www.cs.ucsb.edu/~almeroth/classes/W01.176B/hw2/examples/tcp-server.c .. once you are able to telnet to your server and read/write to your clients, you should be able to progress from there and perform actions when your clients issue a specific command and that sort of thing.

Resources