I am working on project where we may have 1,000's of pieces of hardware in the field all gathering data and reporting it back to a central server.
To get this data from sensor to database the hardware will be programmed with the server address to send it to. Data will be sent over http port 80. A json file will be encoded and pushed out via Ethernet where the server will intercepting, parsing and storing the data from each device based on device location and data structure.
My question is are there well defined ways to configure what I am calling the catcher? What sort of listening socket solutions should I look at?
I have done similar things in the past where data is sent via ftp to a server where I have a cron job (php script) running to look for new files in the destination folder and parse them into a database. This worked for 1 device but I want to do 1,000's.
Thoughts?
Related
The "Real Time Metrics" panel of my MongoDB Atlas cluster, shows 36 connections, even though I terminated all server apps that were supposed to be connected to it. Currently nothing should be connected to it, but I still see those 36 connections. I tried pausing the cluster and then resuming it - the connections came back. Is there any way for me to find out where are they coming from? OR, terminating all connections.
Each connection is supposed to provide with it what is called "app metadata". This is supposed to always include:
The driver identifier (e.g. pymongo 1.2.3)
The platform of the client (e.g. linux amd64)
Additionally, you can provide your own information to be sent as part of client metadata which you can use to identify your application. See e.g. https://docs.mongodb.com/ruby-driver/master/tutorials/ruby-driver-create-client/ :app_name option.
Atlas has internal processes that connect to cluster nodes and cluster nodes communicate with each other also. All of these add to connection count seen on each node.
To figure out where connections are coming from:
Read the server logs (which you have to download first) to obtain the client metadata sent with each connection.
Hopefully this will provide enough clues to identify cluster to cluster connections. You should also be able to tell those by source IPs which you should be able to dig out of cluster configuration.
Atlas connections should be using either Go or Java drivers, if you don't use those in your own applications this would be an easy way of telling those apart.
Add app name to all of your application connections to eliminate those from the unknown ones.
There is no facility provided by MongoDB server to terminate connections from clients. You can kill operations and sessions but connections used for those operations would remain until the clients close them. When clients close connections depends on the particular driver used and connection pool settings, see e.g. https://docs.mongodb.com/ruby-driver/master/tutorials/ruby-driver-create-client/#connection-pooling.
I am trying to find a way to send and receive data over a USB connection connected using a cable. The idea is: A service/command line application will be running on target system, receives commands, and send data for the command. A GUI application sitting on the other machine analyses and interprets the data.
Using libUSB, is it possible?
If you try to connect to PC's together it will not be possible.
The USB protocol is a Master-Slave protocol. Most USB adapters only support either the Master mode or Slave mode. So you will not be able to connect two PC's with USB.
You would have to have special hardware on one of the PC's to act as USB slave.
Several months ago a second exchange server was installed in our domain at a data center, wear-as the first server is located at the local office location.
Both Servers are exchange 2010 SP2 and everybody user outlook 2010 and both have a separate UCC SSL certificate for their own domain.
Both servers have CAS, Hub and Mailbox server roles.
Half of the existing staff members were moved to the new server and are sending and receiving emails via the new server (trusted domain).
The biggest issue for the staff members on the new server is that the shared calendars work very slow. often when opening a shared calendar or setting up a meeting outlook just gets stuck and after half a minute or so and pops up a message "Outlook is trying to retrieve data from Microsoft Exchange server server name.domain." - this mainly happens when attempting to add rooms from the address book and then pops up an error "the operation failed".
Sometimes a message that pops up is:
The address list cannot be displayed.
The connection to Microsoft exchange is unavailable.
Outlook must be online or connected to complete this action.
This is an issue that everybody experiences from the new exchange server. when comparing the the configurations of the two servers noticed that the local auto-discover url of the original EXCH SRV was https://mail.domain.com/Autodiscover/Autodiscover.xml and not https://'netbiosname'/Autodiscover/Autodiscover.xml.
After changing the internal URL for the new server to be the same as the external the performance improved a lot, but there are still many of those popup messages and "spinning wheels". And it seems like lately the problem is just getting as bad as it was before the change.
There are no errors or warnings in event viewer.
I have setup fiddler on my pc and i noticed a number of errors:
There are many 401's when connecting to /ews/exchange.asmx. After several 401, there is a 200.
Outlook connects to the EXCHserver1 autodiscover and not EXCHserverNew - and also gets several 401's before getting to the 200.
When working on an account that is located on EXCHserver1 in fiddler you can just see direct tunnels to mail.domain.com.
When I loaded a mailbox at the data center where the EXCHserverNew is located the performance was much faster but still not 100%, and there were still the same errors showing in fiddler.
But when I tried a EXCHserver1 account at the data center, everything worked just fine.
My last resort to solve this is to move the New exchange server to the local office, which I want to avoid since it wont solve the problem 100%.
I ended up opening a case with Microsoft, and after two weeks of tests with them several things were done to improve the severity of the issue:
Increased the ThrottlingPolicy for the EXCHserverNew cmdlet in EMS
Disabled TCP/IP metrics 'TCP Chimney', 'RSS', 'Autotuning' & ‘NetDMA'
Disabled IPv4 Checksum Offload", "Large Send Offload Version 2 (IPv4)", "Large Send Offload Version 2 (IPv6)", "TCP Checksum Offload (IPv4)", "TCP Checksum Offload (IPv6)", "UDP Checksum Offload (IPv4)", "UDP Checksum Offload (IPv6)", on the NIC card level of the EXCHserverNew
Changed the Hosts file on the clients machines to point ot the local IP on the exchange server.
This improved the issue a lot, but outlook client calender still takes a second or two to load or open the OAB.
Background
I am using a SparkCore wireless arduino board to connect to a local Node.js server. The server includes a local intranet TCP server that a TCP client programmed onto the SparkCore connects to.
Problem
If I run the server on a different network, the server has a different local IP address. When I do this, I have to reprogram the SparkCore arduino to tell it the new local IP address of the server to connect its TCP client to. This is not ideal for a variety of reasons.
Question
Is there a way to have the client dynamically search for the TCP server or alternatively have the server broadcast to TCP clients in a way that would inform the client of the local IP address to use for the server without initially hardcoding it? I would love to do this in way that did not involve iterating through a bunch of IPs on a specific port to see if a connection is made. That being said, if that's the only way to do this, then so be it.
How is the arduino booting? If it's booting using DHCP, one method would be to provide a customer DHCP option that provided the address of the node.js server. ntp, for instance, can configure itself in a similar way. This has the advantage that the arduino need not be on the same local subnet as the node.js server.
An alternative (slightly disgusting) would be to use an A record within your domain (let's say nodejs.example.com. Configure the local DNS recursive server to explicitly return this value (I am presuming you might have lots of different deployments with lots of different nodejs servers).
A third possibility would be to send out some form of discovery packet, either by broadcast, or better by multicast UDP. Assuming it's on the same LAN, the nodejs server could then reply. Clearly you might need to concern yourself with a rogue server impersonating your nodejs server, and therefore might need to add some security (e.g. use a shared secret, send a random nonce plus the nonce hashed with the shared secret to the server, the server checks the hash, and replies with the answer, the nonce, plus the answer hashed with the shared secret and the nonce, each of which the client then checks).
I need to transfer a file between two computers using https protocol using a java code. I have no idea where to start at. Is transfer using socket the only way ? Do I necessarily need to have a code running on the server and one on the client ?
If you already have https server (apache, IIS...) on the machine you want to download from than you will need only the client. Unless your requirement is to write code for both server AND client.
On a general note: YES, in order to transfer file(s) between 2 computers over https you need BOTH server and client.