I have for some time helped a customer to export mdb table data to csv files (and then to further process these csv files). I have used Ubuntu, so mdbtools (mdb viewer) has been available to me. Now the customer wants me to automate the work I do in the form of a Windows program. I have run into two problems:
After some hours, I still haven't found a free tool on Windows that can export my table data in a way that I can incorporate in a program/script. Jackcess (jackcess.sourceforge.net) looks promising, but when running the downloaded jar a totally unrelated Nokia Suite program pops up...
I have managed to open two of the tables in a python program by using the pyodbc module, but all the other tables fail to open because of "no read permissions". Until now I thought that there were no access restrictions on the database, because mdb viewer on Ubuntu opens all tables without any fuzz. There is no other file available to be, just the mdb file. One possibility might be that this is not a permissions problem at all, but a problem with special characters in column names. All the tables that I cannot open have at least one column name with a national character, whereas the 2 two tables I can open do not. I tried to use square brackets in the SQL select called from python, like so:
SQL = 'SELECT [colname] from SomeTable;'
but it makes no difference. I cannot fetch data from the columns that do not contain national characters either (except from the 2 two tables that do work).
If it indeed is a permission problem, any solution must also be possible for my program to perform, there must not be any manual steps.
Edit: The developer of the program that produces the mdb files has confirmed that there is no restrictions for any tables. So, the error message "no read permissions" is misleading. I will instead focus on getting around what I presume is a problem with national characters in column names. I will start with the JSDB approach suggested below. Thanks everyone!
Edit 2: I made a discovery that I feel is important: All tables that I can open using pyodbc have Owner=Admin whereas all tables that I cannot open have no owner at all (empty string it seems, "Owner=").
Edit 3: I gave JDBC a shot. Same error again, as one could expect given the finding in Edit 2. Apparently the problem to solve is the table ownership (although MDB Viewer under Linux doesn't seem to care about that...). Since the creator of the files says he didn't introduce any permission settings, I guess the strange table ownership could be the result of using new programs (like 2010) to read data produced in a old program (like sometime in the 90s), or were introduced during some migration of the old program. Any ideas on how to solve it?
You might be able to use VBScript. VBScript is usually used in ASP files for web pages, but can be used stand alone as a Windows program as well.
VBScript is free as it's code you write in Notepad.
Others may come up with better answers for you. Good luck.
Related
Our current system database system is a clipper DOS application. The database inside its folder is fragmented/divided into many parts. I want to decrypt the database so that I will have only one database in all and avoid reshuffling of data. I'll attached the file folder Screenshot.. the database is on .DBF format
VScreenshot of files
Often you can decompile the CLIPPER exe file to source code and work from the .prg I've done it many times. The program to use is called WALKYRIE.
In Clipper and Fox Pro for DOS .dbf file is a simple table file.
If You want to use as data base with many tables in one unit.
You can import these tables in MS SQL data base and/or part of a MS Access database.
I see that you got several answers. Most are partially right. Let's address these one at a time:
All those files essentially comprise the "database" for the application you're using. They could be used by other applications as well. Besides having a lot of files, what is the problem you're trying to solve?
People mentioned indexes. You can generally ignore these. There are there primarily to make access to the data files faster. Any properly written clipper application will recreate these if they're missing or corrupted. You could test this by renaming one, running the app, and seeing what happens. If it doesn't recreate it you can name it back. Not replacing missing index files would be unusual behavior.
The DBF file format is binary, but barely. Most of what's in a DBF is text and is readable with an editor. But there's no reason to do so - I'm sure there are several free DBF utilities out there to to read DBF files. Getting the structure of the files could be very helpful.
Getting the data out of the files would also be fairly simple with a utility. If you look up the DBF format you could even write one fairly easily in Clipper, any other language that uses DBF files, or in something like Python. Any language that can open and write files, really. It's not hard - any competent developer could do this in a matter of hours. Must less if you're using Clipper or another language that natively reads DBX files.
Most people create dBase/Clipper programs with relational data, like SQL Server. Where SQL Server has tables that relate to each other dBase/Clipper has a file for each "table." This isn't a requirement, but it was almost certainly done this way.
Given that, if you get the table structures through a utility or by reading the headers in an editor (don't save them from an editor!) you could quite likely recreate the database schema (i.e. the map of the data). Once you have that it's fairly trivial to get the data into another type of database (SQL Sever, Access, or whatever you like to use.) If non of the files are too large it's conceivable to put all the files into Excel sheets. It really depends on what you want to do with it.
As others have said, you may be able to get the code by Valkyrie. Some people have used it very successfully. I don't know where you get it and I've never used it. Why do you not have the code? If this is a commercial application you likely should not have it. If it's a custom app who ever wrote it or paid to have it written should have the code.
Again, it's not clear to me what problem you're trying to solve. But there are many options for doing something with those DBF files. Fortunately they are one of the easier to read data formats you could be working with.
Let me know if you have any questions. Apologies for the typos that are no doubt scattered throughout this reply.
You sort of can get an idea of how they relate to each other by opening the index files they use (.NTX files). If you have the DBU utility (executable) around, you can open the DBF and load the index (NTX). LibreOffice Calc is also able to open DBFs (haven't tested .NTX).
If you open the .NTX on a text editor you will see the indexes in the beginning.
I open with Access, but I can save the data using a PrintFill Program.
At work we use a program based on MS Visual FoxPro. Even though everybody uses the same password, and the information inside the program is not very delicate, I haven't been able to get the password removed. Simply because the developers want money to do the job and my boss doesn't want to pay.
I also use a BAT file to open my most used programs and websites, which are pretty much all on auto-login. Except the MS Visual FoxPro program.
I found a BAT script somewhere that waits a certain amount of time, and then mimics keyboards entries. But for some reason it doesn't seem to work on Win10.
So I am wondering if anybody knows a way to automatically sent the password via the BAT file?
The auto-login script I mention above was found here: Automatically open a browser and login to a site?
We use AutoHotkey to automate certain tasks with our own in-house VFP application. It works well. It supports Windows 10 (though we only use it on 7 and Server 2008 here)
So you have an application developed with M$ Foxpro (one of its various versions).
I cannot speak to how the developers 'built' your application. I can only speak for the various VFP applications that I have written.
When I created applications that 'asked' for Username/Password, I compared the input values against VFP Data table field values that were stored away in an encrypted manner so that the casual 'investigator' could not easily determine the values.
That assumes that the user's were allowed to create new Username/Password combinations - thereby requiring support of dynamic entries.
However the application developers could have done it in a variety of ways:
1. Store encrypted Username/Password values into a local VFP data table.
2. 'Hard code' the Username/Password into the application code prior to compilation (most definitely NOT preferred)
3. Run the input Username/Password against a Web Service where these values are stored on THEIR central system.
With those various ways as possibilities - making it more difficult to tell you which way to go, I'd recommend considering the following:
If an issue is BUSINESS CRITICAL, don't quibble over the Dollars.
There may be an answer to this somewhere else on here, but I can't find it.
My organization uses a EHR called TIER that has a SQL back-end. One of the features of the EHR is that you can "scan" a document to a folder on the network with the unique ID of a row in a table on the server. Then from the EHR, you can open a record from the table and then it links to the documents in the folder with the same Unique ID.
An example may be helpful - In the EHR I create a document (a row in the ScannedFormTable) with unique ID of 100. I then "scan" (basically attaching or copying) a pdf or other document into a folder on the network (say D:\ScannedDocuments) with the name of 100, so abc.pdf is now in D:\ScannedDocuments\100. Then from the document in the EHR, I can open the pdf. However, without opening the document to check I can't see if there is any file in the ...\100 folder.
Through some googling, I found that using master.sys.xp_dirtree (and "Undocumented" procedure I think) I can have the EHR "see" the name of files "attached" to the documents. The issue is that I can run this stored procedure from SSMS, but can't from the EHR itself. I have tried to figure out a way to grant security permission for the user in the EHR to run the procedure vs running the script in the background on the server at regular intervals.
Any insights would be greatly appreciated. As you may have noticed from the number of " used, I am a self taught SQL user who is better at googling than actually understanding the intricacies of the language.
I found that using master.sys.xp_dirtree (and "Undocumented" procedure I think) I can have the EHR "see" the name of files "attached" to the documents.
I think you are confusing different things. Yes, YOU can (and tsql can) use that undocumented and unsupported procedure. However, your "EHR" is a system designed to provide specific functionality. It isn't clear what you are trying to accomplish, but to "get" your system to do something obviously depends on your system, its features, and what you are actually trying to accomplish.
I'll add that tsql is not designed to access the filesystem natively - hence the use of undocumented extended procedures. If you are simply trying to verify or traverse this ScannedFormtable table and verify that there is some sort of file in the appropriate location, you might find this task easier to implement in a typical programming language.
If you want better suggestions, it would help to discuss your goal. And consider carefully what you are trying to do, since it is quite easy to create a security hole by altering permissions.
I am trying to split an Access database where I work but I have encountered a few issues that I am struggling to resolve. If I can first explain the problem.
I work for a large multi-national company that has on-site IT support but does not support Access (so no help there)
There are 12 of us working in our section, we have an old and badly designed StockMaster database on the networked F drive. The problem is that it is only set up for single users, we have to take turns using it. We aren't a computer savvy bunch, we tend to run the same named queries on a daily basis
The database is only updated once per day, every morning we get a download from our colleagues in Amsterdam. I do not want to play around with this database as first of all I'm no expert and secondly if I break it, no one will fix it.
My plan is this;
I have created a new Access database StockMaster2 that imports the required tables. Using VB coded modules, is deletes the old then imports the new. Therefore every morning it replicates what is in the original database and it works fine.
My next step is to split the database, create the front end and distribute. This is where I'm having problems.
I created the original front end StockMaster2_fe.accde and placed it in the database folder on the F:\ drive. Does every user get their own copy of the front end? I copied and saved two more front ends (copy and paste in the same folder -> rename) namely StockMaster2_alan_fe and StockMaster2_ryan_fe and tested it. I told Ryan (who sits next to me) to find the front end named after him on the F:\ drive and open it whilst I was in ...alan_fe. We both went to run macros at the same time but he was kicked out as it gave me exclusive access.
What am i doing wrong? Why is it not allowing multiple access?
My problem is that due to strict administrator privileges I cannot download any software or access the command line, so anything that I do must be done in Access itself
I apologize for not seeing this post sooner to end your agony. There are two absolute main issues that must be resolved to get you on the right track. First, and perhaps the most important, is that your file has the name of StockMaster2_fe.accde. The extension, the accde, is the executable version. Design changes cannot be made to that version. The extension should say .accdb to provide you with all the flexibility to alter the database, create one database for back-end tables, and a second database for front-end objects to include queries, forms, reports, macros and modules. If you have the accdb version, then your work will start to get much easier.
Issue number two, if your team is not able to share the database, then that is a sign that the database, when first opening, is opening in Exclusive mode. This option can be changed in the Access Options, in the Advance menu, under Advanced section. Look for Default open mode. It should say Shared to have multiple users operating all at once.
A possible hidden issue that can be happening, is that the database has VBA code which informs the database to open exclusively. With your version of the accde, you will not be able to access that code or change how the database opens.
Let's break this down (only because I finished all my work already...):
My next step is to split the database, create the front end and distribute. This is where i'm having problems. I created the original front end StockMaster2_fe.accde and placed it in the database folder on the F drive. Does every user get their own copy of the front end?
Yes
I copied and saved two more front ends (copy and paste in the same folder -> rename) namely StockMaster2_alan_fe and StockMaster2_ryan_fe and tested it. I told Ryan (who sits next to me) to find the front end named after him on the F drive and open it whilst i was in ...alan_fe. We both went to run macros at the same time but he was kicked out as it gave me exclusive access. What am i doing wrong?
Ensure your back end contains only tables. Access is a "client-centric" database, which means when a query is run it pulls all of the data over the pipe to your local computer, does what it does, and then sends it back. So, make sure the back end has only tables and all the other jazz (macros, queries, etc...) are in the front end. Also, the front end will contain links to the back-end tables. All of your queries/macros/etc.. will reference these links, and not the tables in the back-end DB directly.
Why is it not allowing multiple access?
Also, make sure your table-locking scheme is multi-user friendly. If you're doing table locking, it will cause errors. If you're doing record locking, it probably won't.
My problem is that due to strict administrator privileges i cannot download any software or access the command line, so anything that i do must be done in Access itself.
Shouldn't be a problem at all.
I created a database for tracking metrics, with some automation tricks (email, .doc,.ppt presentations, etc) with a very large Main-table, and lots of forms/GUI. This is the first time I have ever I worried about an MDE/front-end for the thing. So if you would be so kind to answer a few questions, or offer any advice, it would be greatly appreciated (I would hate for all this work to not be utilized).
What is the first thing I need to do? It the 2000 version that must be converted to 03 to create the MDE, but does that get done before I use the database splitter?
Will the amount of objects in the database effect the ability to do this? I have something like 80 forms, 70 queries, 20+ macros, 12 tables, etc...but does the amount of objects prevent some of this from working well once the front end is there?
when i split the database, can I continue to work/make changes and such on the "back end", and have those changes directly effect the front end?
These may be some basic questions, but I don't know the answer so.....Thanks!
Here is my 2 ยข.
Question 1 - I have never used the database splitter as I feel I have more control doing it manually. If you do it manually you can do it to a version that does not have a database splitter. But if you do use the splitter then--yes--you will have to upgrade to a version that has a splitter before doing it.
To do it manually here are the steps.
Backup everything.
Create a copy of your file into the same directory. So if you have an MyApp.MDB create a copy into the same directory with a new name, such as MyAppDATA.mdb.
Open the new DATA file (MyAppDATA.mdb) and delete all of the objects EXCEPT the TABLES.
Open the App file (MyApp.mdb) and delete all of the tables.
Also in MyApp.mdb...go to the File/Get External Data/Link Tables menu to link the tables in MyAppDATA.mdb to MyApp.mdb. Select All and create the links.
That should do it. And if you screw up you made a backup...right?
A couple of tips and gotchas...be sure that you go to Tools/Options and that you are NOT showing System and Hidden tables. You just don't want to delete system tables from MyApp. Another way to do it is do NOT delete tables that start with MSys or USys.
Question 2 - Does not matter how many object you have. In fact you don't have that many objects anyway.
Question 3 - Yes...you will make backend changes in MyAppData.mdb and when you open MyApp.mdb those changes will auto-magically be there to see and query against etc. (In the query designer you may need to save/close/reopen to see new fields if you made the mod while in the query). The EXCEPTION to that is New Tables You will have to use the File/Get External Data/Link Tables option to create links to new tables.
One thing to remember (and that I hope you already realize) is that the one downside of splitting the database is that when you deploy the front end file that usually the relative path to the data will vary from machine to machine and there is no automatic re-linking of tables in access. If your target clients have full access you can always use Tools/Database Utilities/Linked Table Manager to refresh the links to the right location. If you can't do that then you will have to do one of the following:
1. Write code that does the automatic re-linking for you. Basically it will check the links...if invalid it will prompt the user for the data location (or look it up in an INI file) and re-link the tables.
2. Always deploy your app to the same location on all machines. If you have commercial visions for your application this won't work...I mention it for academic reasons. It might be doable for a limited deployment where you have a lot of control over file placement on each machine.
3. Put the Data file (MyAppDATA.mdb) onto a network share and link the table across the network using a drive mapping or UNC (\myserver\mydata\ApplicationData\MyAppData.mdb). The latter is preferred but both of them run the same risks as number two.
Seth
PS This answer assumes Access 2003.
PPS If you have commercial visions for your application then the table linking has got to be REALLY robust.
PPPS I agree with the commenter that you may want to take the plunge and do SQL if it is in your skill set.
One thing that hasn't been discussed, and that's the issue of whether the compile to MDE could fail. Basically, if your code compiles in your front-end MDB, it will convert to an MDE. But I've noticed that lots of people never compile.
Some hints for keeping your VBA code in good shape:
in VBE options, turn off COMPILE ON DEMAND.
add the COMPILE button to your standard VBE toolbar and USE IT OFTEN.
periodically, backup your MDB and decompile/recompile it.
Also, remember that you must keep the MDB source, as the VBA code is not editable in an MDE and not recoverable by any good method.
EDIT:
Steps for a decompile:
backup your MDB.
start an instance of Access with the /decompile commandline argument. For, instance, I have a shortcut on my deskstop that has this as the target:
"C:\Program Files\Microsoft Office\OFFICE11\MSACCESS.EXE" /decompile
having opened that instance of Access, open the MDB you want to decompile. You will see nothing happen. DO NOTHING FURTHER IN THIS INSTANCE OF ACCESS -- close this instance of Access (the reason for this is that Michael Kaplan, who knows a thing or two about this, recommended that you never do any work in an Access instance opened with the decompile switch because he said there was no guarantee that the Access application code executed under those circumstances in a way that was fully safe for all kinds of Access work).
open the just-decompiled MDB holding down the shift key (you want to be sure that startup routines don't run because that would likely recompile the product before you've finished your cleanup) and compact the MDB (holding down the shift key again).
open the code editor and compile the project (DEBUG -> COMPILE [db name] for those who haven't step #2 in my original compiling instructions at the top of the post before the edit).
compact the MDB (doesn't matter if you bypass startup, since it's already fully compiled).
Why so many steps?
Because the purpose of the decompile is to get rid of the compiled p-code in order to start afresh from the canonical VBA code. Following the steps above insures that you have completely cleared the data pages storing the compiled code before you recompile. The reason for this is that without the compact step after the decompile, under some very rare circumstances, the code can behave strangely. I can't imagine that the old discarded p-code is being used again, but there's something about the pointers between the canonical code and the compiled code that apparently doesn't get completely flushed by a decompile without a compact.
This would be a comment to Seth's answer, but my rep isn't high enough to comment yet.
Seth did a great job answering your questions, I just wanted to add a bit more to part #1 about using the Database Splitter. The Database Splitter in the Tools menu works fine. Doing it manually is alright too, but it's a whole lot faster and easier to use the Database Splitter. I've used it a dozen times and never encountered any issues after using it.
http://www.databasedev.co.uk/split_a_database.html has a decent page about some of the pros, cons of splitting your database.
http://www.accessmvp.com/TWickerath/articles/multiuser.htm also has some good info when dealing with a split database in a multi-user environment.
Seth gave you a very good answer. But I'll add a few comments.
The number of objects only becomes relevant when you get close to about 1000 forms, reports and modules which have code. There's a limit about there. If you do get that message when trying to make an MDE then you almost certainly have a code error and need to compile to find the error
Another resource is "Splitting your app into a front end and back end Tips"
See the Auto FE Updater downloads page to make the process of distributing new FEs relatively painless.. The utility also supports Terminal Server/Citrix quite nicely.