I have just started using TB for about a week. I have installed TB in Ubuntu 20.04 with postgresql DB. Created dashboard with dummy data and also real data from sensor attached to Raspberry Pi.
The dashboard is showing telemetry data fine. But my question is how to store these data to Postgresql DB, tables? Seems there is option in Rule Engine to direct the data to DB, but how the Rule Engine node will identify which device is sending what data. There could be many devices under one customer/tenant. And in DB, which table? Would like to see past data from DB (say months or years).
Searched net for sometime, but didn't get any concrete information. Any help would be very much appreciated.
Thanks in advance.
Related
I started exploring Databases and PostgreSQL not long ago, my goal is to create a database for my company, in which I can upload data and other people can access it through ODBC connections and explore it in Excel. I managed to create a test database in my PC and everything worked properly.
I then moved to the next step, which would be sharing this test database with my colleagues.
I guess that the database could not be stored in my own PC (as it is today) , because people need to be able to access it even when it is turned off.
We have a Network Drive, where currently all our data is stored. Is it possible to host my database in there? What else should I know for this project I'm conducting?
I'm quite new to the subject and I'm just trying to explore some possibilities.
(We all run Windows)
I am working on a project where i need to display the database mssql server's performance metrics for example memory consumed/free, storage free space etc. I have researched for this purpose and one thing came up was DOGSTATSD.
Datadog provides the library for .net project to get custom metrics but that was not the solution for me because the metrics works on datadog website. I have to display the all (in graph or whatever suited) data, received from MSSQL SERVER. There will be multiple servers/instances.
Is there a way to do that, our WebApp connected with multiple databases and we receive/display information.
I cannot use already available tools for the insights.
You can easily get all needed data via querying dmv and other resources inside SQL Server. Good start is here.
I have an EC2 box which I SSH to often. I want to install a redis database in the box and query that frequently. Unfortunately, the code to store values in the database can not be run on the box since it gets 'killed'.
So I have been thinking of running the code on my personal laptop and storing the values to the Redis database on the box. So I need to query the database in the box again and again which is not on my laptop but that is on EC2.
Appreciate your help and time.
Thanks!!
I am required to setup a web application that will interact with an existing ERP system (WinMagi). The ERP is basically a front-end to an xBase (FoxPro) database. The database is located on an in-house server. The ERP, as far as I'm aware, doesn't have an API but can accept purchase orders, etc through an EDI module. The web application should be able to accept online orders and query data for reporting.
My plan so far:
Synchronize the xBase DB to a SQL server instance on a cloud hosted VM.
(one-way from ERP -> SQL Server)
Use this sync process as an interface between the ERP and web application.
Push purchase orders back to the ERP using EDI.
My thinking here is that it would be safer from a data concurrency perspective to create or update data in the ERP through a controlled and accepted (by the ERP) interface.
Questions/Concerns:
What is the best way to update the SQL DB from the xBase DB? Are there any pre-existing libraries that can do this so I don't have to reinvent the wheel?
Would the xBase DB become locked during sync? Or otherwise cause an issues for the live ERP?
How do I avoid data concurrency / integrity problems during the sync?
This system wouldn't be serving live data to the web app. What sort of issues can I expect due to this?
Should I prefer one language over another for this sort of project? My plan was to use Java/Hibernate MVC.
Am I perhaps going about this the wrong way? Would I be better off interfacing my web app directly with the xBase DB? Some problems that immediately spring to mind with this approach are networking issues between the office and the cloud-based VM and potential security vulnerabilities from opening up the ERP directly to the internet.
Any advice or suggestions you might be able to provide would be greatly appreciated!! Thanks in advance.
UPDATE - 3 Sep 2012
How I'm currently doing the data copy (it's not a synchronization) - runs nightly:
A linux box in the office copies the required DBFs from a read-only share on the ERP server to local storage.
The DBFs are converted to CSV using Dave Burton's fantastic dbf2csv perl script
The resulting CSVs are rsync'd to the remote VM. There are only small changes in the data so this is quite fast.
Once the rsync is complete the remote VM does a mysqlimport to the production DB.
Advantages of this approach
The ERP cannot be damaged in any way as the network access is read-only.
No custom logic has to be implemented to sync data and hence there are no concerns that the data could be wrong on the remote VM.
As the data copy runs at night the run time isn't too important.
Current run time is approx 7 minutes for over 1 million records with approx 20-30 fields per record.
Longest phases are the DBF copy and conversion to CSV.
Disadvantages
The DBFs have to be copied in full every time.
The DBFs have to be converted in full every time.
Tables that are being copied are locked during the mysqlimport. This isn't really too much of an issue though as the import runs during the night and the mysqlimport only takes about 20 seconds.
If you are using Visual Foxpro 3.0 or greater, you could use the built in DataBase container to create a connection to the SQL Server DB. Then the Views in the .DBC would do the heavy lifting of reading and updating the SQL Server tables.
I would envision a routine that looped through your Foxpro table and reading the rows and then making the updates to the SQL Server DB. So, the Foxpro tables shouldn't be lock. To ensure this, you could first query the DBFs into a cursor, then loop through the cursor.
I would suggest adding procedure to do concurrency checking.
Another option to server live Foxpro data in your web apps would be to create a linked server in SQL Server to your Foxpro database. That way your Foxpro data could be accessed real time.
I am currently doing something similar - I have to make invoice transactions from a FoxPro-based system available through a web application that will be on a remote, hosted VM running SQL Server.
I will answer your first point based on what I'm doing - you can decide for yourself whether it would work for you!
What is the best way to update the SQL DB from the xBase DB? Are there any pre-existing libraries that can do this so I don't have to reinvent the wheel?
I didn't really look for any shared libraries. What I did was (somewhat simplified):
Added a field to the ERP-side transaction table that holds a CRC32 value based on other fields that I want to detect changes to (for example, the transaction balance).
Wrote a standalone EXE that scans the ERP-side transaction table on a timer, calculates a CRC32 value based on some fields, compares this to the last CRC32 value stored in the new field from point 1, and if different then something has changed and the transaction needs to be re-sent. This EXE was written in VFP for simplicity in accessing DBF files, and it runs as a Windows service. When I get time it will be re-done in C#.
Still in this EXE, once I have a list of new or changed transactions I convert them to JSON. I rolled my own JSON functions, but you could use Craig Boyd's from [Sweet Potato Software][1] or a number of others. There may be a PDF document associated with the transaction, if so it is encoded and embedded in the JSON.
I send the JSON to a web service on the remote side using a class that leverages the standard Windows WinHTTP library (WinHttp.WinHttpRequest.5.1) . The remote web service is essentially running Java. It decodes it all and updates the SQL Server.
HI all,
I had created a app(a Product to be sold to many users) which is almost complete, now I have a tough task to sync the app with the website, which will be used to manage the data for all users. So it is quite obvious I will have to update the database that will reside on web server from the users desk through the app it self. Lets say I just want the users personal data to be stored on the DB on web server and anytime they update it in the app that should reflect on website as well(meaning the data on DB on webserver should be updated)
ANy ideas for doing this in best possible way, also this is just the example I will need to update heaps of data from users desk. I am already streching my hair thinking of managing data on server, can any one please advice here.
Thanks and Regards
Nirav
You can write a update engine to upload users data to web server database.I had a exprience with Sql server replication on windwos XP and have some trouble and final I decided to develop a windows service that can do it.