We have a system based on AcuCobol and vision-files.
We need to dump the entire datafiles in to txt-based files to be imported in to a sql server.
We are using vutil to dump the data right now, but it's creating a fixed-width file and we would want this to be a delimited file of some sort. The command we are using right now is this:
vutil -unload -t sourcefile destinationfile
Now does anyone have any experiance with this, and if so, what would be the best utility for this?
what I'll do to get the job done is this:
Unload your table with a JCL into a file.
Use this file in a batch,
with a copybook matching the description of the table :
(ex: if your table has 3 columns [ID], [NAME] and [ADDRESS], then create a copy CC-1 with:
O1 CC-1
05 ID PIC X (16).
05 NAME PIC X (35).
05 ADDRESS PIC X (35).
For each line move the line into the copy CC-1.
Create a second copy CC-2
O1 CC-2
05 ID PIC X (16).
05 FILLER PIC X VALUE ';'.
05 NAME PIC X (35).
05 FILLER PIC X VALUE ';'.
05 ADDRESS PIC X (35).
05 FILLER PIC X VALUE ';'.
Do a move corresponding of CC-1 to CC-2
Write in the output.
This for each line. You get in the output a CSV file of all your data.
You can reuse the batch and JCL, you just have to change the size of the records and the descriptions of the copy to match the table you are unloading.
Free and easy. :)
Google is your friend... I did a quick search and came up with NextForm which claims to convert Vision to CSV (and other formats).
From the "fine print" it looks like you can download a trial copy but will have to fork out about $400.00 to get a licenced version (that won't drop random records).
I have absolutely no experience with this product so cannot tell you if it really does the job or not.
Edit
Sometimes it is more productive to work with rather than fight against a legacy system.
Have you looked at AcuODBC? This lets Windows programs read/import
Vision files. You could potentially use this to access Vision files from any Windows ODBC enabled application. I believe you may need to compile a data dictionary to make this work, but it could provide a bridge between where you are and where you want to be.
Another approach might be to write your own dump programs in AcuCobol. Read a record, format and
write it. Writting a Cobol program of this complexity should not be too challenging a task provided the files
do not have a complex structure (eg. multiple record types/layouts in the same file, or repeating fields within a
single record).
Your task will be significantly more complex if a single Vision file needs to be post-processed into
multiple SQL Server tables. Unfortunately, complex conversions are fairly
common with Cobol legacy systems
because of their tendancy to use rich/complex/denormalized file structures (unlike SQL which works best with
normalized tabular data).
If the Vision files are more complex that a single SQL Server table, then maybe you could consider using the
AcuXML Interface
to dump the files into XML and then use an XSLT processor to create CSV or whatever other format you
need from there. This would allow you to work with fairly complex Cobol record structures. Again,
this approach means writting some fairly basic AcuCOBOL font end programs.
Based on your comments to my original post, you are not a die-hard Cobol programmer. Maybe it would be
a good idea to team up with someone in your shop that has a working knowledge of the language and
environment before pushing this any further. I have a feeling that this task is going to be a bit more complicated than "dump and load".
If you have Acubench under the Tools Menu there is an option for Vision File Utility, from there you can Unload your Vision Data to a text file which is tab delimited.
From there you can import to Excel as a tab delimited file and then re-save as a csv file.
Related
Often I have to write individual SAS programs for various demands where the final output is an EXCEL file. For the sake of reproducibility I often copy and paste the SAS code into a separate EXCEL sheet -- so it is easy to get back to the point when there is a similar request or the need for modifications.
Now I am wondering if there exists a way to make SAS export the current code into an EXCEL sheet in an automated way. Such a task is probably easier if the SAS code has been saved as a physical file on the hard-drive (so it not only exists in the working memory) -- so let us assume that this was indeed the case.
If the program has been saved as a file, then you can read it into a dataset (the same way you read any other file) and export that dataset to Excel. It's up to you how exactly you do both of those things, SAS gives you lots of options.
As far as telling which program is executing, it varies by how you're running things. If you are using DM SAS and the Enhanced Editor, you can use this code:
%let filename= %sysget(SAS_EXECFILEPATH)\%sysget(SAS_EXECFILENAME);
%put &=filename;
and now &filename. holds the current program's path and filename.
If you're running in batch, you have to use the environment variable SYSIN. Other modes have different methods, and it's not always possible (such as if you're in an embedded program in an EG project). See this doc page for more details.
I am making a c version of pacman and am keeping my highscores in a separate .txt file called highscores.txt. In the program, after the completion of a game, it checks to see if the highscores file should be updated and writes over it if it should. The high scores view within the program reads the scores and names from the file. The issue is that it is very easy for someone to simply edit the .txt file and say they got any score they wanted. Is there a way to make it so that the file can only be written on by the program? This is in a Linux Red Hat environment.
I'd say just encript the file, then rename it something obscure.
An easy way will be to gzip it, and name as a file called "data" or whatever, so people couldn't guess it's gzip.
This method is easy to break once you know gzip is used, so a more secure way is to encript it using an encription key internal in your code.
Your best option is probably encryption. A good, quick study on Encoding vs. Encrypting vs Hashing
You could also look at other ways of storing the data. For example, something like HSQLDB or SQLLite, where you can create databases that have usernames and passwords available.
I come to you completely stumped. I do some side work for a company that uses an old DOS based program to input and retrieve data. This is a legacy piece of software, and they have since moved to either QuickBooks or Outlook for all of their address or billing related needs. However there have been some changes made, and they work with this database fairly regularly. Since the computer that this software is on, is running XP (and none of the other computers in the office can run it) they're looking to phase this software out for when the computer inevitably explodes.
TLDR; I have an old .csv file (roughly two years) that has a good chunk of information on it, but again it's two years old. I have another file called ml.dat (I'm assuming masterlist.dat) that's in the same folder as this legacy software. I open it with notepad and excel and am presented with information like this:
S;Û).;PÃS;*p(â'a,µ,
The above chunk of text is recognized much less within notepad or excel. It's a lot more of the unrecognized squares.
Some of the information is actually readable however. I can for example read the occasional town name, or person's name but I'm unable to get all of the information since there's a lot missing. Perhaps the data isn't in unicode or something? I have no idea. Any suggestions? I'm ultimately trying to take this information and toss it into either quickbooks or outlook.
Please help!
Thanks
Edit: I'm guessing the file might be encrypted since .dat's are usually clear text? Any thoughts?
.DAT files can be anything, they are usually just application data. Since there is readable text, then it is very unlikely that this file is encrypted. Instead you are seeing ASCII representations of the bytes of other content. http://www.asciitable.com/ Assuming single byte values, the number 77 might appear in the file somewhere as M.
Your options:
Search for some utility to load and translate the dat file for that application.
Set up an appropriate dos emulator so you can run this application on another box, or even a virtual machine running freedos or something.
Figure out the file format and then write a program to translate the data.
For #3, you can attach a debugger to the application to trace how the file is read and written. Alternatively you can try to figure out record boundaries (if all the records are the same size, then things are a little bit easier.) Then you can use known values to try to find field boundaries. If you can find (or reverse compile) the source code, then that could also give you insight into the file format.
1 is your best bet, and #2 will buy you some time so that you don't need that original machine anymore. #3 would likely be something to outsource.
If you can find the source or file format, then you just recreate whatever data structure was dumped to the file and read the file into it.
To find which exe opens it, you can do something like:
for %f in (*.exe) do find "ml.dat" %f -c
Assuming the original application was written in C then there would be code something like this to read the first record from the file:
struct SecretData
{
int first;
double money;
char city[10];
};
FILE* input;
struct SecretData secretdata;
input = fopen("ml.dat", "rb");
fread(&data, sizeof(data), 1, input);
fclose(input);
(The file would have been written with fwrite.) Basically you need to figure out the innards of the SecretData structure to be able to read the file.
There likely wasn't a separate utility used to make the file, dumping data and reading it back from a file is relatively easy in most languages.
I would like to create (or use, if one already exists) a command-line based app that creates, modifies and searches a database.
This database would ideally be a simple text file where each line is an entry.
As in this simple example:
Apple Fruit Malus Green/Red 55
Banana Fruit Musa acuminata Yellow 68
Carrot Veget. D. carota Orange 35
Let's say this text is stored in ~/database.txt
I'd like to be able to search for all entries that are of the type fruit (returning, Apple and Banana) or all entries that have kilocalories that are less than 60 (returning Apple and Carrot) on the command line.
The returns should happen through standard terminal output and should look like this:
$mydatabasesearch cal '<60'
Apple Fruit Malus Green/Red 55
Carrot Veget. D. carota Orange 35
Also, being able to add to the database through the command line would be awesome!
Is there anything around that does this? If not, how would you recommend I write such an app? I know a bit of C++ but that's it...
Take a look at sqlite. It is a bit more complex than plain text files, but a lot more powerful.
Plain text files are not really considered databases.
If you want to stick to textfiles and the commandline, have a look at the usual unix utilities like grep, awk, and the coreutils package (cat, cut, uniq,...) which work on plaintext files. Add those commands to a shellscript and you're done.
As soon as you move to some sort of database system you won't have textfiles as storage anymore.
Aside the already mentioned sqlite, the Berkeley DB library might also be worth a look if you want to write your own program. Both libraries should be good in your case since they don't require an external database server (like mysql)
You can do this in Unix anyway with a delimited file SED and maybe some simple commandline perl which you can wrap in a bash script. There is a good tutorial for the data manipulation at wikibooks and the SED you could use is probably all here and in the part two tutorial.
I am building a basic POS app for my cousin's pharmacy store so that he can dump the software he is currently using and save on license cost.All the medicines name which he has painfully entered into the software have been stored in a file with .d01 extension.
What i want is a way to read the contents of the .d01 file programmatically so that i can import the name of the medicines into my app.
The s/w from which my cousin uses is built in Foxpro(coz i see a lot of .cdx,.idx,.dbf files) and the file which i want to import is with .d01 extension. when i open the file in notepad it is something like this
http://img192.imageshack.us/img192/5528/foxpro.jpg
So I assume its somekind of database table or something. So can anyone please help me in reading this file, as i am not at all aware of foxpro.
Thanks a lot in advance to all those who take out time to reply.
hey guys thank you very much for replying so promptly.. I tried the solution suggested by Otávio and it worked, i will now write a small utility to read the dbf.
It has a good chance of being just a regular .dbf file. Copy it somewhere safe, change the extension to dbf and see if you can open it from foxpro.
Although it may have .cdx files, the actual paste of the file does not appear to be a visually recognizable header format of a VFP table... even if part of a database container. The characters around each column name don't look right. It may be from another language that also utilized "Compound Indexes". I even saw an article about Sybase's IAnywhere too. If worst-case scenario, and it is determined to be a possible fixed-length per row and no dynamic column sizes, you might take the file, strip off what appears to be the header and leave just the data and stream read it in based on how many constant characters are determined for the length. yeah, brute force, but just an option. Again, it doesn't LOOK like a VFP table.
BTW, what is the name of the software he is using... I'd look into that to see if any other type of indication to its source.
It looks sort of like a DBF file - maybe Clipper or something.