I have a client that uses a point-of-sale solution involving an Access database for its back-end storage. I am trying to provide this client with a service that involves, for SLA reasons, the need to copy parts of this Access database into tables in my own database server which runs SQL Server 2008. I need to do this on a periodic basis, probably about 5 times a day.
Is there an easy programmatic way to do this, or an available tool? I don't want to handcraft what I assume is a relatively common task.
I am running this on SQL Azure, so there's no way for me to run prepackaged software on the server. It would either have to be open source and portable to Azure or executable on the client's computer.
I'm unfortunately thinking I'm going to have to roll my own tool to do this. Any suggestions or more tools that are out there that can do this themselves before I go ahead?
David, I looked at multiple solution for a similar problem: converting from dbf to mysql, here are 3 solutions (all commercial - but relatively inexpensive) that can work for you:
Full Convert
SQL Manager
ESF
Other than that I couldn't find a good robust data conversion tool that would be open source or free. At least not for DBF to MySQL conversion. There might be something out there for SQL/Access. You could roll out your own solution, but is it worth your time?
DISCLOSURE: I ended up using Full Convert.
Also all of these products generate some sort of batch file, that can be scheduled using Task Manager.
There are two things to consider:
connectivity
ETL tool
For connectivity, you will need to establish VPN tunnel of some sort between the client server and your server.
Then use SSIS to connect to MS Access, to create packages to pull data from MS Access to SQL Server database. On SQL Server, you will need to create new schema, to mirror or be close MS Access
On connectivity side, another option - since MS Access db is in the file, you may be able to FTP the file to your server and point SSIS to the file
Related
We have several MS projects being used and a system written in SQL Server by us. Would MS Project Server allow us to set up something like SSIS procs to continuously get data from the MS Project databases and insert them into our SQL Server database?
Is there a way to link all systems or a better way to do this where we're not paying a monthly fee for MS Project Server? Would actually like to bypass MS Project Server if possible.
Thanks
Yes it is possible. You can write a vba routine which sends the schedule data up to a database. I've done this previously sending schedule data up to SQL server using ODBC.
You'd need to decide what data you want to see - getting task, resource and assignment data is quite straightforward, but if you need to look at time phased data that can slow things down. Then it's just a case of looping through the collections and using a SQL INSERT statement to squirt each task, assignment or resource up into the tables.
If that's all project server was being used for, then you can retire it and its supporting infrastructure. Additionally, you would then only need Project Standard client, which is about half the price of the Project Professional client.
For SQL Server, we are able to send over the db for the most part pretty easily to offshore staff.
Is this possible with the AS/400 or they can only VPN in to work?
Every database engine has a slightly different version of SQL. DB2 for i at V5R4 has differences to DB2 LUW 9.7 and both are different to SQL Server and MySQL at any version. So the quick answer is no, you can't simply make a copy of a DB2 for i database and run it on MySQL or SQL Server. You'd normally do exactly as you are doing with SQL Server: Have one machine here and another machine there and unload/reload the data as needed.
Having said that, the differences between SQL dialects are not usually crippling. Use the IBM Navigator for i and extract all of the DDL for the IBM database, then try to execute the DDL on the SQL Server machine. You'll have some syntax problems, but you should be able to work them out with someone who is knowledgeable in both dialects. Keep track of the changes to the DDL because you'll need them in order to extract the data from the IBM side.
Once you have the empty database created on the new machine, it's time to extract out the data. Write some CL programs to do CPYTOIMPF to generate CSV files or flat files or whatever it is that SQL Server wants in order to import properly. Then FTP that data to the new machine and write some scripts to do the import.
As you can tell, this is not going to be a simple process and it will take some time to develop and debug. I'd go with having the offshore staff using a VPN to the local IBM machine.
The easiest way I can think of would be to create a Save File (SAVF) then FTP that save file to the other IBM i and [restore it] (http://pic.dhe.ibm.com/infocenter/iseries/v6r1m0/index.jsp?topic=/cl/rstobj.htm).
In the PC world this is similar to zipping up a directory, FTPing it to another machine and then unzipping it.
If this isn't what you mean, can you elaborate on what you're wanting?
The offshore site probably has their own SQL Server, probably running the same version as you.
But unless they also have an IBM Power System running the same release of IBM i, then they will most likely need to access your system.
I'm looking for the best approach (or a couple of good ones to choose from) for extracting from a Progress database (v10.2b). The eventual target will be SQL Server (v2008). I say "eventual target", because I don't necessarily have to connect directly to Progress from within SQL Server, i.e. I'm not averse to extracting from Progress to a text file, and then importing that into SQL Server.
My research on approaches came up with scenarios that don't match mine;
Migrating an entire Progress DB to SQL Server
Exporting entire tables from Progress to SQL Server
Using Progress-specific tools, something to which I do not have access
I am able to connect to Progress using ODBC, and have written some queries from within Visual Studio (v2010). I've also done a bit of custom programming against the Progress database, building a simple web interface to prove out a few things.
So, my requirement is to use ODBC, and build a routine that runs a specific query on a daily basis daily. The results of this query will then be imported into a SQL Server database. Thanks in advance for your help.
Update
After some additional research, I did find that a Linked Server is what I'm looking for. Some notes for others working with SQL Server Express;
If it's SQL Server Express that you are working with, you may not see a program on your desktop or in the Start Menu for DTS. I found DTSWizard.exe nested in my SQL Server Program Files (for me, C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn), and was able to simply create a shortcut.
Also, because I'm using the SQL Express version of SQL Server, I wasn't able to save the Package I'd created. So, after creating the Package and running it once, I simply re-ran the package, and saved off my SQL for use in teh future.
Bit of a late answer, but in case anyone else was looking to do this...
You can use linked server, but you will find that the performance won't be as good as directly connecting via the ODBC drivers, also the translation of the data types may mean that you cannot access some tables. The linked server might be handy though for exploring the data.
If you use SSIS with the ODBC drivers (you will have to use ADO.NET data sources) then this will perform the most efficiently, and as well you should get more accurate data types (remember that the data types within progress can change dynamically).
If you have to extract a lot of tables, I would look at BIML to help you achieve this. BIML (Business Intelligence Markup Language) can help you create dynamically many SSIS packages on the fly which can be called from a master package. This master package can then be scheduled or run ad-hoc and so can any of the child packages as needed.
Can you connect to the Progress DB using OLE? If so, you could use SQL Server Linked Server to bypass the need for extracting to a file which would then be loaded into SQL Server. Alternately, you could extract to Excel and then import from Excel to SQL Server.
A colleague I work with recently told me that SQL Express and MS Access were essentially the same thing; that does not seem to be an accurate statement. I know you can convert Access to a SQL DB and maybe under the covers they are similar, but I would assume that the SQL DB engine and what is used to run access are not the same. Not only that, but the SQL statement syntax, etc. I know are not the same.
I am mainly trying to understand so that I am more informed about the versions.
Um, no, not the same.
First off, I need to clear up some terminology. MS Access is a Rapid Application Development (RAD) tool that allows you to quickly build forms and reports that are bound to relational data. It comes with a file-based database engine (Jet/ACE).
Access the RAD tool can be used with many different backend databases (Jet, SQL Server, any db that supports ODBC, etc). I have to assume your colleague was specifically commenting on Jet/ACE, ie the database engine that MS Access uses.
I think the single biggest difference between the Jet/ACE database engine and MS SQL Server Express is that Jet/ACE is file-based and SQL Server Express uses a client/server model. This means that SQL Server Express requires a running service to provide access to the datastore. This can complicate deployment in some scenarios.
SQL Server Express is really just a throttled-back version of SQL Server: max database size of 4GB (10GB in 2008R2), only uses a single physical CPU, etc. These limitations are imposed to prevent large organizations from using the freely available Express edition in place of a full-blown SQL Server install. The upshot to this is that SQL Server Express offers a truly seamless upgrade path to SQL Server. It is also (generally speaking) a more robust and fully featured database management system then Jet/ACE.
Similarities
relational database management systems
written by Microsoft
Differences
MS Access
File based
free distributable runtime (2007 or later)
RAD tools (form/report designer)
uses Jet SQL
max file size 2GB
SQL Server Express
Client/Server model
free
no RAD tools
uses Transact-SQL
max database size 4GB (10GB for SSE R2), max one physical CPU
I think what your colleague had in mind was SQL Server CE, which is a super-lightweight embedded database, which is still (IMO) far superior to Access in database-management aspect. SQL Express cannot even be compared with Access without offending the former.
Here are the datasheets for both products so you can see some hard facts on the difference between the two databases.
Access:
http://office.microsoft.com/en-us/access-help/access-specifications-HP005186808.aspx
SQL (Express is listed on the far right column):
http://www.microsoft.com/sqlserver/2008/en/us/editions-compare.aspx
The comment I have always read is that Access is great for single user single access database use, the minute you scale beyond a single user look elsewhere. While that may be a "bit" of a stretch, Access really does not do well in a multi-user environment. From experience we've had a client who has ignored and ignored our requests to migrate a backend database from Access to SQL, and there have been numerous occasions where we have had to restore from backups, or take the Access database offline due to corruption.
They are two completely different technologies with two different target markets. The database engines are indeed different, as you mention T-SQL is different than Access SQL.
You can "scale up" an Access database to SQL by creating an SSIS package or other tool to do the import, but this takes the Access schema and data and migrates it to a true SQL database. It does more than just attach the Access database or the like.
Anytime you need a "real" database I'd highly recommend looking at any of the SQL versions that are available over Access.
Just remember that with MS-Access you don't have size limitations if you play your cards right. There is no reason, for example, not to have many 2 to 4 Gig tables each contained singularly in their own database. Your ODBC applications can open a connection to multiple MS-Access databases and query the single table in each. So you can have a database containing trillions of records, stored in multiple MDB files. One company I went to work for was using a single MS-Access database to run a issue tracking system done in MS-Access forms. They could only use it one person at a time because of sharing issues that would lock MS-Access up. I wrote a Win32 Perl native Windows GUI user-interface to the database that was better at field/record validation, and my ODBC code was able to manage the connection for simultaneous user access. I managed the opening and reading and writing and closing of the database for each user through my Perl program. I did not leave the database open. I did not maintain a persistent connection for each user, but instead only maintained a connection long enough to retrieve a record for edit. Then I closed the connection until it was time to write the record back to the database. Also, I wrote my own record locking program logic by maintaining a user login table that contained the record id of the record a user was currently editing, then erased that entry when no longer editing that record. When another user went to edit the same record, the program checked if that record was currently open for edit by another user. The system worked flawlessly. MS-Access never locked up via ODBC and multi-user access. I even embedded the password to the database in my compiled Perl program so that no one could get to the data in the Access database other than through my Perl program.
Hello,
I'm new here, so sorry, if my question is too basic. However, maybe you have some advice, example, links, which could help me... I'm trying to find something helpfull for few days, but no results as for now.
I'm working in a distributed environment. I have a Oracle server hundreds of miles away and a MS SQL server close to me. I'm writing a application using Visual Web Developer 2008 Express. I need some data from Oracle. It's not worth to query the Oracle server every time i need some data from it. I'd prefer to run some Oracle queries once each night and store results in some local (SQL Server) tables. I assume, I should run queries through standard windows scheduler (Windows Server 2008). I have the basic connectivity - I can open Oracle Database from local Visual Studio.
The questions are:
How to write a query/procedure/function that would get data from Oracle and put them into a SQL Server table (possibly recreated before each query run)?
How can I run such a query from command line (or in other way run from scheduler)
What naming conventions are applicable? In VS I use something like //IP.IP.IP.IP/Name and a user with password.
Thanks for any help or advice.
Regards,
Matteo
I suggest you speak to the DBA's of the Oracle and SQL Server databases, as there may be other considerations you need to bear in mind. (Data Integrity, Security, ownership etc.)
One route you could follow would be to implement DTS (For older databases) or SSIS (for new versions of SQL Server) processes to copy the data across on the schedule you want. (This is pretty much what they were built for.)
How much data are we talking about?
If there is a small quantity that you need to transfer every day, you can write a stupid fetch and insert script in language of your choice.
You only need to search for better solutions if "sync" would take too much resources.
Thanks...
I'm the DBA for the SQL Server, which will serve only for my application. For Oracle I just want to read data and I have enough privileges and agreement with DBA's. Security, ownership and integrity are not an issue for now. I just need some technical advise how to get data from Oracle to MSSQL tables on a schedule.
I use MS SQL Server 2008 Express SP1. I'm very close to solve my problem - I have established connections and everything installed and working. I just don't know, how to run a query, which would get data from Oracle and put into MSSQL, on regular basis, without manual interaction.
I've some experience in programming, but not much in databases (except creating complex SQl queries). Therefore some example or links to detailed description would be helpful. I'm not sure about naming conventions, differences between procedures, functions and queries, command line options to run db automation procedures and so on. I'm also not sure, about which mechanisms or technologies are available in MS SQL Server 2008 Express edition.