I'm working with the data of opencellid, I do have in my database all the data from Brazil, but now it is necessary to keep this updated. To do this, I want to use an API in Python which every day goes to opencellid and get the diff data (OCID-diff-cell-export-2021-03-26-T000000.csv), but I only was able to find APIs which give me information from a specified cell. So, how can I keep my database updated using only APIs?
Related
I work for a small publishing company with an internal website that displays a static HTML table of our published products.
We have a need to be able to list and sort published products (about 1-2 items are published per day) dynamically that is being fed from an Excel spreadsheet. The Excel spreadsheet is what we are currently using to maintain the data. The Excel spreadsheet is on a shared network drive available to the company.
I am familiar with AngularJS, ReactJS, and VueJS2 for front-end development and was wondering if I would be able to use one of those tools to consume a Excel file, parse it to JSON, and then display it dynamically on the client side.
Is something like this is possible?
When a user finishes editing the Excel sheet and saves it to the shared network drive, is there a script that would automatically save the data as JSON? I assume we would then simply have our Javascript framework reference and consume the saved JSON to populate its published products list.
Note: We are unable to use a relational database at this time (ie MySQL).
Part 1 - generating json from excel...
front-end technologies are not the way to go. You need to run a service that watches folder for change (like nodejs or python). Saving as csv instead of xls might make things easier as you may not need extra libraries to make sense of your xls file
Part 2, displaying json data...
Your browser, by default, cannot load a local json file. So you may need to run a server (again nodejs and python make this relatively easy) to host your json file.
there are many ways of presenting data these days, but without knowing some of your particular and based on the information you did share, looks like you've got a steep learning curve to get something like this going.
There is a requirement in my project where we need to design a system which can collect data through Web API and then use the data to compare and copy the received data to an existing SQL Server DB. I want to know if anyone has already worked on such requirement and if yes then what is the best way to design it? I am currently thinking of below two options. Please let me know which one is better and if there is any other option.
My algorithm will be as - fetch the data through web api -> compare the data -> save mismatched data to a particular table -> copy new data to the existing tables.
The two options I am currently thinking of are-
1) Use a windows service which will run once in a day and execute above algo.
2) Use SSIS package which will run once in a day and execute above algo.
If anyone has used either of this solution, please guide me to an article or blog which can be helpful to me.
I have similar project requirement before. What I achieved is in SSIS.
Brief steps:
Using C# script to get the return data (http://json2csharp.com/ is an easy way to return C# class based on your JSON components)
using third party dll, install Newtonsoft.Json to deserialize the JSON
Assign the results in C# script to each pre defined variable (be careful with the data type)
Compare the result with the existing table in data flow task.
Let me know if you have any questions
i am developing a website on angularjs and my server gets information from a graph db in neo4j. At first ive used the default neo4j db (with movies and such) but when i want to load my own csv files the neo4j adds onlyhalf of them. I have 117000 rows and ive tried to use periodic commit and it added again, only 58000. What is the cypher command for adding all the data, is it ok to divide it to another csv file?
EDIT: Ive used this command:
USING PERIODIC COMMIT
LOAD CSV FROM 'http://docs.neo4j.org/chunked/2.1.2/csv/artists.csv' AS line
CREATE (:Artist { name: line[1], year: toInt(line[2])})
Another question: i need to show the result from a query using angularJS and i couldnt find a normal explanation, algorithm, example etc. is there a way to show the result (the result is in json)
EDIT: i need to show the results as a table and also as a nodes (like in the neo4j admin website)
Yes, please split it up into 3 questions.
For loading data,
see http://jexp.de/blog/2014/06/load-csv-into-neo4j-quickly-and-successfully/
And check out MERGE:
see: http://docs.neo4j.org/chunked/milestone/query-merge.html
For creating an angular application with Neo4j backend
see: https://github.com/kbastani/neo4j-movies-template
I am a complete newbie when it comes to web page design, and what I am trying to achieve is a web page that I can display on a wall mounted screen as an office dashboard. I have data in excel that is constantly being updated (on the server) and I want to be able summarise this and display it (e.g. total orders etc.) for staff to see. Therefore the web page needs to be able to connect to the data source, and update itself every few minutes. I am hoping to then use Ubuntu or even Raspberry Pi to drive the dashboards.
Can anybody point me towards either some clear instruction on how to achieve this, or better still some sample files that will help me see how its done?
Really appreciate any help!!
If you want to use PHP, you can use PHPExcel, to read your Excel files, and if you want to only display the information as is, you may output it to HTML directly, there is an API to manipulate the Excel file, so you may use that to summarize your data, however, if you need something cleaner, you may want to use Windows instead of Linux, given that in Windows, you can use an Excel file as a data source, and there are third party products that can use them as if it were a database (using SQL queries to retrieve data).
I am currently working on completing my Masters Thesis project. In order to do so I need to be able to obtain the raw data accumulated in NagiosXI and/or OPSview. Because both of these are based off of the Nagios core, I assume the method to obtaining the raw data may be similar. This raw data is needed so that I can at a later time perform specific statical calculations which relate to my Masters Thesis. I have looked online and so far found some Nagios plugins which obtain raw data and then manipulate it for graphs and visuals, but I need the raw numbers in order to complete my calculations.
I am also researching to see if I can create maybe a PHP script, or some other language, that will extract the data from Nagios and save it in a word or excel document. However, this would be a bit of extra work as I am unfamiliar with either PHP or MySQL queries. Because of this I hope to be able to find a plugin, or something similar, that can get the data for me.
Cyanide,
I can't speak for NagiosXI, but I can for Opsview :)
You could access the data that is stored in the RRD files. You can use rrdtool dump to pull the values out or use a URL like: /rrdfetch?start=1307608993&end=1307695393&hsm=opsview%3A%3ACheck%20Loadavg%3A%3Aload1&hsm=opsview%3A%3ACheck%20Loadavg%3A%3Aload5
And this returns back the JSON data points. This is undocumented, but is used to power the interactive javascript graphing.
Alternatively, if you have ODW enabled with full statistics, then the raw data is stored in the ODW database and you can then extract the raw data with SQL commands. See http://docs.opsview.com/doku.php?id=opsview-community:odw for more information.
Ton
You can try use mk livestatus http://mathias-kettner.de/checkmk_livestatus.html
or http://exchange.nagios.org/directory/Addons/APIs/JSON/Nagios2JSON/details
All this tools get you status data without need to go to DB or status file. While XI is based on Nagios it can still work with him.
Please take a look at http://dmytro.github.com/nagira
It's a web services API to access Nagios data. You can get all hosts, service status data, objects configuration in multiple formats JSON, XML or YAML.