I am working on a website/model which will predict weather forecast of a specific place namely Kothri in Madhya Pradesh. But I am not getting any datasets for it. Can someone help me to get some information regarding this? And what are the available methods and resources for obtaining historical weather data for this, specifically for the period of the last 20-30 years? Please elaborate stepwise.
I was trying to find datasets from IMD but couldn't get from there.
Related
I know this question is fairly simple for you and there are many related answers but none of them explain all what I need.
So you all must have been to YouTube, there you must have seen the Trending Posts Section, where all the videos which have the most views/likes/dislikes (I don't know exactly) w.r.t the time are listed. Now, all those videos are from different different channels.
I want to know:
There must be an algorithm or function inside the server to do that? If there is one, then when do it runs, i.e, are the results kept ready inside the server the same a google search engine --- which keeps the pages stored according to their ranking and whenever someone searches it delivers the result.
The above algorithm goes through every posts, right? But when? How does it knows that this post from this youtuber is getting this amount of views or a lot of views.
Now, after it knows which posts are trending --- where are they stored. Like, if we take mongodb database. Is it kept in a different collection like trending posts or their id's
I know the above questions must be confusing because of my half(or no) knowledge. But, please help me know it all. I am trying to get this answer for a very long time. Any guides or tutorials appreciated. Thanks!
For a "trending posts" feature you would have to store each view/like/etc (whatever you consider as a metric for "popular"). You also need to store a timestamp, so that you can select for "current popularity" as opposed to "in the past". And maybe store some identity of the viewer, so that each visitor is only counted once.
Next you have to find out which post is trending at the moment. For this you would need to find out the posts with the highest number of views in the past timespan. It is up to you (and the nature of the system you are running) whether you use hours, days, weeks or even months.
This query is probably too expensive to run on each pageview, plus the results don't change that much from minute to minute. So you would run this on some schedule, for instance once every hour. Again, it is up to you decide how accurate you want/need it. Maybe run this several times an hour, or just once a day. You do need to store the list of "trending posts" somewhere, so you can display it easily.
This is my first time in the forum so if I made any mistakes please let me know.
So I have to do a research work for school and I am having a little bit of a problem finding a data set that meets all my requirements. This is for my database class. I have been looking the whole weekend and I have not been able to identify any valid source.
Still after all that I have been able to find nothing.
I was wondering if you could help me; perhaps there is a more flexible website to help me narrow the search; or a specific website with the information..
Here are all the requirements:
The data set must be from a legitimate source (e.g., the us gov, a state agency, a university).
The data set must measure something by date and by zip code. in essence, the data set can contain just 3 fields (date, zip code, measure).
The date range must be at least 10 years and span.
The level of granularity of the date must be at least by month, and in the format of YYYYMM or YYYYMMDD.
Zip code must cover all fifty (not 58) states.
Anything except weather
Thanks a lot in advance for the help
You can try the U.S. Census Bureau, there are several datasets that should match your need, although there is probably more information than the minimum you need: that is likely to be true in most real-world examples.
Okay. I was looking at my physio's spreadsheets for his small, private business today. He uses Excel to keep track of his clients appointments, fees, attendences, medical reports etc. At the moment he has a single sheet where he adds every clients appointment to the list as he goes - there's 3 years of details, one row for every appointment! It's huge and pretty hard to navigate and make sense of when he's extracting information such as fees paid/unpaid, total visits, etc.
I'm a novice to sub-intermediate at Excel, but getting better. What I'm wondering is it possible to set up a "front page" where he can enter a day's details in a single spreadsheet, press an export cell,and then have Excel pass the relevant data to individual sheets for each client. The data on that front page would look for the clients name as a string to find the relevant sheet and drop the information in.
I'm not asking how to do it as such, but rather wondering if this is possible at all!
Thanks
=COUNTIF(Sheet1!RangeToLookThrough, ValueYoureLookingFor)
Put this where you want.
If you want a daily dashboard, you could replicate this to do something like :
=SUMPRODUCT(--(Sheet1!B2:B4000<>""),--(MONTH(Sheet1!B2:B4000)=9))
Feel free to alter this to capture yesterday or whatever date your looking for. I just used an old report to mess with and give you an example.
I am writing my Bachelor thesis about Market Basket Analysis and I need a data set to make an example of this analysis, can anyone recommend me something?
It would be very good if data would be big enough, for example around 1000 rows or more and with names of items purchased not just numbers...
Any help would be very helpful!
For others who are seeking for dataset related to market basket,I found dataset in kaggle interesting
https://www.kaggle.com/puneetbhaya/online-retail
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
Where can I find historical raw weather data for a project I am doing with focus on the USA and Canada. I need temperatures mainly, but other details would be nice. I am having a very hard time finding this data. I really dont want to have to scrape a weather site.
I found myself asking this same question, and will share my experience for future Googlers.
Data sources
I wanted raw data, and lots of it... an API wouldn't do. I needed to head directly to the source. The best source for all of that data seemed to be either the NCEP or NCDC NOMADS servers:
http://nomads.ncdc.noaa.gov/dods/ <- good for historical data
http://nomads.ncep.noaa.gov/dods/ <- good for recent data
(Note: A commenter indicated that you must now use https rather than http. I haven't tested it yet, but if you're having issues, try that!)
To give an idea of the amount of data, their data goes all the way back to 1979! If you're looking for Canada and the US, the North American Regional Reanalysis dataset is probably your best answer.
Using the data
I'm a big python user, and either pydap or NetCDF seemed like good tools to use. For no particular reason, I started playing around with pydap.
To give an example of how to get all of the temperature data for a particular location from the nomads website, try the following in python:
from pydap.client import open_url
# setup the connection
url = 'http://nomads.ncdc.noaa.gov/dods/NCEP_NARR_DAILY/197901/197901/narr-a_221_197901dd_hh00_000'
modelconn = open_url(url)
tmp2m = modelconn['tmp2m']
# grab the data
lat_index = 200 # you could tie this to tmp2m.lat[:]
lon_index = 200 # you could tie this to tmp2m.lon[:]
print tmp2m.array[:,lat_index,lon_index]
The above snippet will get you a time series (every three hours) of data for the entire month of January, 1979! If you needed multiple locations or all of the months, the above code would easily be modified to accommodate.
To super-data... and beyond!
I wasn't happy stopping there. I wanted this data in a SQL database so that I could easily slice and dice it. A great option for doing all of this is the python forecasting module.
Disclosure: I put together the code behind the module. The code is all open source -- you can modify it to better meet your needs (maybe you're forecasting for Mars?) or pull out little snippets for your project.
My goal was to be able to grab the latest forecast from the Rapid Refresh model (your best bet if you want accurate info on current weather):
from forecasting import Model
rap = Model('rap')
rap.connect(database='weather', user='chef')
fields = ['tmp2m']
rap.transfer(fields)
and then to plot the data on a map of the good 'ole USA:
The data for the plot came directly from SQL and could easily modify the query to get out any type of data desired.
If the above example isn't enough, check out the documentation, where you can find more examples.
At the United States National Severe Storms Laboratory Historical Weather Data Archive (note: this has since been retired).
Also, the United States National Climatic Data Center Geodata Portal.
The United States National Climatic Data Center Climate Data Online.
The United States National Climatic Data Center Most Popular Products.
wunderground.com has a good API. It is free for 500 calls per day.
http://www.wunderground.com/weather/api/