Automate daily collection of weather station data from NCDc - database

Here is the link to where the data can be found. Scrolling down to Surface Data : Daily and selecting the Global and U. S. Web Page Icon. On the next page, Select the radio button Country and press continue. From the next page's drop down menu, select any state and then press the radio button for entire state (I need more than 100 stations per day).
The next page allows you to select the date range.
I am dumping these data files into excel and sorting (which is a pain, since it could obviously be
automated).
Is there a way I can access the most current data via a script (VBA or otherwise)?
My main problem is that I can't send the server a simple request to obtain the data. Also, there is an FTP site but only allows .gz files to be downloaded, each over a gig, so its not reasonable for my situation.
Any help would be appreciated!

There National Climatic Data Center has several data APIs available for this.

Related

Pop up window with data from another spreadsheet

I'm trying to build a master spreadsheet in Google Sheets to call up data (not import data, just make a quick view in a popup window) for individuals from other spreadsheets. E.g. my master sheet has a column for therapists, a column for certifications, a column for periodic reviews, etc. All of these columns correspond to other sheets within the project that list out the certifications for each individual therapist, the periodic reviews for each individual therapist, etc.
I want to be able to click in the cell where "Mary" intersects with "Certifications" and see a small popup window that shows me Mary's record in the Certifications page.
I tried a couple of functions pasted from other pages (it's been a long time since I've coded), but none worked. This seems like a simple database-like call, so I hope the solution is also simple. I appreciate any help offered. (My clinic doesn't have an in-house IT person, so I'm trying to help where I can.)

React router - different content if post in list vs own page

React router has very cool feature: if you have a list of items (Instagram for example) and if you click on one item, content opens in modal/overlay but if you copy-paste the link to new tab/window or share the link with a friend for example, it opens in its own page.
I would love to use this feature but I need to find a custom solution..
My posts are very data heavy
I've split posts data into 2 tables in database
1st is very lightweight containing essential data: 4-5 columns
2nd table is very heave, ~30 columns
When user uses search filter, list updates only with data from 1st table
If user clicks on post, it will open in a modal/overlay
I will recycle the data I already have (from 1st table) and also get rest of the data from 2nd table
However, when user shares the link or opens it in new tab/page, data from 1st table is not present. I would need to integrate a conditional logic:
If post opens in list view (modal/overlay), only get additinal 2nd table data
If it's opened in a new tab/window in its own page, get all the data, 1st table included
How could I integrate this with React router? Has anyone already tried it? This would also allow to use different layout/components when user opens item in page view. Is there a way to check it?
Or is there a flaw in my logic? I imagine list would update very fast because it doesn't require huge amount of data and also would modal/overlay because it recycles some of the data.
I read all the docs, also searched online - didn't find anything.
Modals in react router are great. I've used the pinterest example and adapted it to my own needs.
Ensure you do your check on state.modal===true in a master layout component to give you the modal styling.
you'll need to check if table 1 stuff is present in your state and dispatch an action to trigger the async call in componentDidMount. You should be fetching table 2 in all scenarios.

How can I generate counts for items with differing view permissions in my database without being inefficinet?

My web-app has a media upload facility which associates each uploaded media item with a collection of media items (for simplicity we'll just refer to each group of media items as an album, though the encompassing entity for media items is not as simple a structure). Each of these "albums" has the potential to have custom viewability settings which specify who can view the album (a la Facebook).
I'm trying to generate a count of media items uploaded by a user for display on the user's front page.The problem is, I can't figure out a way to do this without perusing through the data of all the "albums" (in essence, its doing the same amount of work required to collect the viewable albums).
If all the albums had the same viewability settings, I could just maintain a count of the media items and manipulate it on upload & deletion. Since they can potentially all have different viewability settings, however, I have to check the viewability permission of the viewer for each "album" to make sure its media items can be included in the count that will be displayed to him.
Consider Facebook:
Upon looking at a user's profile, you can see a count of photos uploaded by the user. Presumably this count is of all photos you're allowed to view, and not the total count of the user's photos. If this is true, it would be inefficient to peruse through all the album database rows (and rows in other tables, such as those containing the "Lists" of allowed viewers) just to generate a count; you still have another click to get to the page where you can view the photos (where a permission verification process will have to be completed again)!
Is there something I'm missing, or is this now sites handle items like these? Perhaps the process is completed the first time and the viewable albums are cached for subsequent use?
I don't know about Facebook, but... Unless you're doing something extremely complicated with the permissions, if you have a well-normalised database and properly set up indices, the join should not be that slow. Can you show the relevant schemata?
After momentarily blocking a friend from viewing some of my photos, then viewing the profile as him (using the "View as" option) at the top-right corner, it appears as though the photo count doesn't change. In other words, the count is the value of all the photos uploaded by the user, regardless of the privacy settings on each.
I'd be grateful if anyone could confirm this.

How to get specific texts from the URL put on MS Excel cell?

I want to build the database which gets the url from user in one cell and returns some texts to the other cells. The Url is from www.imdb.com. I saw it was possible on Google Spreadsheet. I am wondering if it's possible for MS Excel.
Like when I input a movie Url, it gets ratings,title, casts from IMDB page.
You can get pretty close using a IQY file to define the web query and the parameters a user should input. The only difference is that, rather than defining the movie id in a cell, your user will be prompted when they create or refresh a query.
To do this, you need to:
create a plain text iqy file. Call it "imdb-movie.iqy" or something.
Put this text in it:
WEB
1
http://www.imdb.com/title/["Movie ID"]/
Selection=3
Formatting=None
PreFormattedTextToColumns=True
ConsecutiveDelimitersAsOne=True
SingleBlockTextImport=False
DisableDateRecognition=False
DisableRedirections=False
In the "Get External Data pane of the Excel Data ribbon click "Existing Connections" and find your IQY file. It will ask you where to put the data and prompt you for the movie id. By right clicking, you can see lots of settings about formatting and such.
The "selection=3" entry in the IQY file determines what table to pull the data in from. It sounds like you may want a number of them.
To make your own IQY file, you can choose the "get external data/from web" and select which table to use. Then choose "save" from the dialog and you have the file. You can paramaterize it with the ["parameter name","prompt"] syntax.

VB.net Winforms dropdownbox data load question

I have a VB2005 winforms application that will loads city data from my database table. This is to ensure that the user enters the correct city spelling, in order to receive an accurate quote. Currently, there are about 150K cities that are being loaded to the dropdown listbox on page load. It takes about 30-40 seconds for that page to load.
My initial thought was to allow the user to select the state first. Then load the city values. But the user has the option of going back and requesting a quote for a different city / state.
Is there a more efficient way to handle this?
Quotes are based on state and city name? If so, your approach sounds good, but consider basing quotes on zip instead, or zip/city name. Not everyone will find their city in your 150K list, and some don't live in named cities at all.
For starters you could have a BackgroundWorker or Thread load the data in the background into an array or list. You'd then pass this data to the comboBox when needed.
If you use this method, you must find a way of reindexing the fields from time to time though.

Resources