I'm going to build an enterprise grade database on the topic of securities. I read on the SEC website that there is an CIK code the government uses. I suppose there are also other identifiers. Anyone done this recently? Can you please point me out to where I can download lists of securities and their identifiers?
It sounds like you want a list if all companies. I believe this is what you want, from the SEC Edgar database
http://www.sec.gov/edgar/indices/fullindex.htm
Related
I asked this over on Superuser and it was suggested I try this here:
Can anyone recommend a quality source to learn databases? I am changing careers and have no background in computers but this is what I have chosen to do now.
I was thinking of taking an intro course at a community college but I have no problem teaching myself with a book and some software. I am looking to accelerate my learning curve and don't want to spend an entire semester on an introductory course if there is something better out there.
Any suggestions are welcome. Thank you.
Edit:
Thank you for the feedback. The MS site and Oracle look promising.
I am pursuing a career in software development. I have taken C++ and C# at a community college and was accepted to a masters program for the spring. What level of database knowledge/implementation is required to program in C++ at the master's level and not get handed your lunch?
I guess I need to know how databases are used in programming. I don't have the common core of experience in order to explain the question more thoroughly without tangentially departing into illusory ideals of a programming career.
At any rate, what you have provided is enough to get me going and it will, I am sure, uncover further areas of interest.
Thank you kindly.
There's a whole lot of aspects of "learning databases":
administering databases
designing databases (modeling)
implementing databases (might require adding business logic into triggers and stored procedures).
using databases (and this might be 'writing a query' to aspects of data mining)
And then there's what type of database. Most people these days assume RDBMS, but there's the push towards NoSQL and tuples stores, etc.
There's no one good answer without a little more information about what you're trying to do.
In addition to what people here would consider as database-related jobs (DBA, database developer, data modelling analyst), there are a lot of semi-technical jobs around that refer to themselves as "database people". In particular, you get report analyst -- often using tools like Crystal Reports, Cognos, SSRS -- and business analysts who work on large customer databases, often defining more how the data should be used in various marketing activities than doing much coding. So it would be up to you to decide if you wish to have a more "business" aspect to your career goal, or to work in data entry/management, or do development and design, or to become a specialist admin certified in a particular vendor's software.
In all cases, however, given you start from scratch, some understanding in how relational database systems work would be beneficial. Someone mentioned Element K courses. Personally, I've found the SQL Zoo interactive online course particularly engaging, and have used it with new staff who had to get up to speed with SQL skills in a short time. Also, this tutorial looks well done. For self-study, I recommend the Head First book series, as in Head First SQL. All these recommendations are extremely SQL-centric, though.
Microsoft also offers tutorials that go with Access and SQL Server Express, but I'd warn not to become locked in. SQLite is a free and extremely lightweight RDBMS, which is perfect for trying things out, either using a programming language (if you know one -- if not, I recommend Python).
As for recommending a provider, you're not even saying which country you're in (though from mentioning "community college" I guess it's the US), what your budget and mobility are like etc, so it's hard to decide. Maybe a mentor or advisor at your community college could help?
The IEEE Computer Society has some online Element K database courses available for members. IEEE runs memberships by the calendar year. Professional membership dues are $50 for a half year and $99 for a full year of professional membership. Student memberships are $40 and $20 for full and half year, respectively. This might be less than the cost of one course.
ACM has some online database courses in their catalog also. They also use Element K, so it seems likely they are the same courses. A full membership is also $99 per year, student is $19 per year. (Their membership year starts when you join.)
I assume your goal is to get a job, since you mention you are changing careers.
I understand you wish to accelerate your learning curve. Make sure you do lots of hands-on work, then, to make sure you know how to do things and have skills in databases. The skills, as they say, pay the bills.
I don't know what your background in computer programming, computer science, or information science / managed information science is. If you have some background into those topics, you might be able to make it into the stuff I recommend below.
Most databases are relational in businesses. Depending on the business, they run different datbases. The two really big ones are Oracle and SQL Server.
I've known a couple of friends who learned databases (and other enterprise software) by going online and reading as much documentation as they could put their hands (mouse?) on the Oracle and SQL Server websites, and for other enterprise software. They went and applied for jobs, had enough business sense / charisma, could read/write American business English well, and were good enough to get the job with little/no experience, and have done very well. It pays their bills, and is a stepping stone to bigger things. You might be able to do the same. Go on the Oracle or Microsoft websites, and grab the documentation. Read as much of it as you can, circle words you don't know, try to find the definitions. Find a local guru (that's the main advantage of the community college, they have smart people with communication skills who can help), and talk with them.
If your a tremendous reader, you could do this in a month.
EDIT:
As to how much "database" knowledge you need to know to get into your master's...
It depends on the program; I'm going to assume your'e going for a professional program and not a academic one. Depending on the master's program, you may/may not need to know databases. Consult your professor or advisor to see if this is something you could tackle right away.
You need to know, ultimately, how to "talk" to a database. Think of this like trying to do business with folks in a foreign country:
You'll need to learn the language (relational databases use a language called SQL; make sure you know how to formulate a basic query (SELECT), and modify the data in the database (INSERT / UPDATE / DELETE)).
Make sure you know the culture (what is a table, what is a row, what is a column, what is a tuple, what are keys (foreign/primary/candidate/alternate/super), what are triggers, what are the normal forms, what is an ERD diagram, what is relational theory)
Make sure you have safe transportation (learn APIs to communicate with the database in C++ and C#. Learn how you can run SQL queries on the database from your app and get resulting data back inside your app. Perhaps learn about ORM tools and how to get/set data that way).
The above is what an intro course covers.
I'd like to create an in-house solution to store marketing segment, list, campaign, and communication data. Right now nothing is centralized/standardized. Data is located on a variety of SQL servers, Access databases, and Excel spreadsheets. It's been a real pain when it comes to reporting/tracking.
I'm in a Microsoft SQL Server environment and have access to:
Microsoft Access
Microsoft SQL Server Management Studio
Microsoft Business Intelligence Development Studio
Security and compliance is pretty restrictive in my environment. Purchasing a third party software package doesn't appear to be an option. I may have the possibility of having a SQL server sandbox environment created for my use.
I'm curious what suggestions you would recommend and why. I need to think about all aspects including existing data retrieval/parsing (some on a continuing basis), data import into the new marketing datamart, and reporting. Some kind of GUI may be required as there isn't currently one for tracking/categorizing much of the data. One other individual may need access to help with regular imports to help spread workload.
Thanks.
Your requirements are incomplete:
existing data retrieval/parsing (some on a continuing basis),
data import into the new marketing datamart, and reporting.
Some kind of GUI may be required as there isn't currently one for tracking/categorizing much of the data.
One other individual may need access to help with regular imports to help spread workload.
Here's your question: "I'm curious what suggestions you would recommend and why"
Here's the answer.
Who will use this? Exactly who? Call every single one of them and talk to them about what they do.
What is the business value? "solution to store marketing segment, list, campaign, and communication data" is a bad idea. No one wants a "solution" -- they want to get their jobs done. Few people have "problems" that need "solutions". They are already perfectly able to do their job. The best you can do is make them more efficient. Do they care about their personal efficiency? I doubt it.
Who has the problem? What problem do they have?
Think of a "Bacon and Eggs Breakfast". The chicken lays an egg and walks away. The pig, however, is totally committed to the bacon.
Find the pigs and chickens. Your data is not going to identify your actors and your business problem. Find the people who have the problem. Find out how big and costly the problem is. Be absolutely sure, you've found the real problem that is costing someone real money. Cozy up to the person who's losing the most money and make sure they want their problem solved. They are the pig -- they can be totally committed.
Eventually, you'll want to build one central SQL/Server database and get rid of Excel Spreadsheets and MS-Access databases. You might want to have an MS-Access front-end which provides nice applications that use SQL/Server.
If you are actually talking about a data warehouse, then you must next read books by Ralph Kimball. [It's not clear, BTW, that a data warehouse is even relevant. With no problem to solve and no user who has the problem, a data warehouse is just as bad an idea as a web services framework or a new Bentley for me (Black and Silver, thank you.)
If you want to build a data warehouse, your "existing data retrieval/parsing" and "data import into the new marketing datamart" will called ETL.
Your "and reporting" will be moved from an "oh, by the way" to the central most important feature of whatever it is you're doing.
Your "Some kind of GUI" will go away. Data warehousing is not much of a GUI thing. Reporting is as close as you get. Maybe you'll have to create some Master Data Management tools, but even then, those are more rules than interactions.
"One other individual may need access" Really? The end users are what? Chopped liver? They need query access, too, or they won't see any of your data.
You indicate that a purchase of third party software is not an option, however, you should consider that the purchase may in fact be cheaper than the in house development effort, and may bring you to a working system much faster, with less in house skills required.
I've got 10 GB of files to back up daily to another site. The client is way out in the country so bandwidth is an issue. Does anyone know of any existing software or libraries out there that help with keeping a folder with its files synchronized across a slow link, that is it only sends files across if they have changed? Some kind of hash checking would be nice, too, to at least confirm the two sides are the same.
I don't mind paying some money for it, seeing as how it might take me several weeks to a month to implement something decent on my own. I just don't want to re-invent the wheel, here. BTW it is a windows shop (they have an in house windows IT guy) so windows is preferred.
I also have 10 GB of SQL Server 2000 databases to go across. Is the SQL server replication mode reliable?
Thanks!
Look into rysnc.
This might be what you're looking for seeing as you're an MS shop
Microsoft Sync Framework File Synchronization
Two more suggestions are Robocopy and XXCopy.
I'm part of a software development company where we do custom developed applications for our clients.
Our software uses MS SQL Server and we have encountered some customers which do not have a DBA on staff to manage the databases or if they do, they lack the necessary knowledge to perform their job adequately.
We are in the process of drafting a contract with one of those customers to provide development services for new functionality on our software during the next year, where they have an amount of hours available for customization of our software.
Now they want us to include also a quote for database administration services and the problem is that they are including a clause that says that those services will be provided only when they request it.
My first reaction is that db administration is an ongoing process and not something that they can call us once a month to come for a day or two. I'm talking about a central 1TB+ MSSql Cluster and 100 branch offices with MSSql Workgroup edition.
My question is for any suggestions on how I could argue that there must be a fixed amount of hours every month for dba work and not only when their management thinks they need it (which I’m guessing would only be when they have a problem).
PS: Maybe this will be closed as not programming related. But I'm a programmer and I have this problem. My work is software development but i don't want to lose this client and the only solution I can think of is to find a way for the client to understand the scope so we can hire a qualified DBA to provide them with the service they require.
Edit: We are in a Latin American country with clients in the Spanish speaking region. My guess is that in more developed countries there is a culture that knows how delicate the situation is.
This is definitely one of those 'you can lead a horse to water, but you can't make them drink' situations.
My recommendation here would be to quote the DBA services as hourly, and make the rate high enough that you can outsource the work if you decide you want to. When (not if) the SQL servers start to have problems, the firm is on the hook.
I would also recommend that you include in your quote a non-optional 2 hour database technology review once per year. This is your opportunity to say 'You spent XXX on database maintenance this year, most of which was spent fighting fires that could have been easily avoided if you had just spent XXXX/4 and hired a DBA. We care about you as a customer, and we want you to save money, so we really recommend that you commit to using a DBA to perform periodic preventative maintenance'.
I would also recommend that you categorize any support requests as having root cause b/c of database maintenance vs other causes. This will let you put a nice pie chart in front of the customer during their annual review (which they are going to pay you to perform). It is critical to manage the perception so they don't think your code is causing the problems. You might even go so far as to share these metrics (db related issue vs non-db related issue) with them on a quarterly basis.
Sometimes people need to experience pain before they change. The key is to not be in between the hammer and their thumb as they learn the lesson, and hourly quoted work is one way of doing this.
As a side note, this sort of question is of great interest to a large number of developers. I'd say that this sort of thing could impact a programmer's quality of life more than any algorithm or library question ever could. Thanks for asking it!
No DBA on a system that size is a disaster waiting to happen. If they don't understand that, they are not qualified to run a database that size. I'd recommend that they talk to other companies with similar sized databases and have them ask them about their DBAs and what they do for them, and if they think they could survive without them.
Perhaps the link below from MS SQL Tips could give you some good talking points. But people who aren't technical wont respond to a technical explanation of the necessity of good DBA you are likley going to have to work toward proving the cost of bad DBA. Work out the worst case scenarios and see how they feel about them. If you can make it seem like a good financial move (and I think we all know it is) it will be an easy sell.
http://www.mssqltips.com/tip.asp?tip=1278
A good and flexible database schema illustrating e-commerce products and their rates management is required.
I agree with Chris Roberts, OSCommerce is a good place to start as well as DatabaseAnswers.org.
Hope that helps.
I'm sure there are many examples about - try looking at some open source shopping cart tools.
As a general point, though, I think there are so many ways that a product pricing and discount / gift voucher system could work that finding a database schema that works for every possibility is unlikely!
Perhaps you can save yourself some heartache by refining the requirements a little?