I have multiple CSV files from different sites within my company that contain multiple names and email addresses from several SQL databases that many different company users enter data into. I have a SQL export program that exports the names and emails to CSV files. I have noticed that occasionally some of the email addresses have the hex 0x1F separator either before or after the email address (in Notepad++ it looks like a black "US" box).
How can I write a simple batch file to find and remove just that separator from any CSV file that may have it. And save the output using/replacing the original file. Preferably using simple batch commands, not Powershell or Java or anything else like that. This will be running on a standard install of Windows 2008 R2 without any extra programs added.
Example:
Directory:
C:\Uploads
Filenames (up to 23 files with a random 2 digit prefix followed by date):
"a1-20151101.csv", "b2-20151101.csv", "cd-20151101.csv", etc.
Inside CSV (FirstName;LastName;Email):
John;Doe;john.doe#johndoe.com
Jane;Smith;jsmith#google.com
You could spend precious time writing some 400-line behemoth of a cmd file.
Or you could simply go and get the tr program from GnuWin32 (ports of the popular UNIX tools to native Windows) which is perfectly suited to doing this sort of thing.
Then your batch file will basically consist of the line:
tr -d "\37" inputFile >outputFile
The tr program is contained within the coreutils package.
Related
I need to store data in a file in this format
word, audio, jpeg
How would I store that all in one file? Is it even possible do would I need to store links to other data files in place of the audio and jpeg. Would I need a custom file format?
1. Your own filetype
As mentioned by #Ken White you would need to be creating your own custom file format for this sort of thing, which would then mean creating your own parser type. This could be achieved in almost any language you wanted but since you are planning on using word format, then maybe C# would be best for you. However, this technique could be quite complicated and take a relatively large amount of time to thoroughly test your file compresser / decompressor, but may be best depending on your needs.
2. Command line utilities
Another way to go about this would be to use a bash script to combine all of the files into one file, and then decompress it at the other end. For example the steps could involve:
Combine files using windows copy / linux cat command on command line
Create a metdata file of your own that says how many files are in this custom file, and how much memory each one takes up (could be a short XML or JSON file for example...)
Use the linux split command or install a Windows command line file splitter program (here's just one example) to split the file back into whatever components have made it up.
This way you only have to create a really small file type, and let the OS utilities handle the combining of them for you.
Example on Windows:
Copy all of the files in your current directory into one output file called 'file.custom'
copy /b * file.custom
Generate custom file format describing metadata (i.e. get the file size on disk in C# example here). This is just maybe what I would do in JSON. SO formatting was being annoying so here's a link (Copy paste it into an editor or online JSON viewer).
Use a decompress windows / linux command line tool to decompress each files to the exact length (and export it back to the exact name) specified in the JSON (metadata) file. (More info on splitting files on this post).
3. ZIP files
You could always store all of the files in a compressed zip file, and then just use a zip compressor, expander as and when you like to retreive any number of file formats stored within.
I found a couple of examples of :
Combining multiple files into one ZIP file in only C# .net,
Unzipping ZIP files in C#
Zipping & Unzipping with only windows built-in utilities
Zipping & Unzipping in Linux command line
Good Zipping/Unzipping library in Java
Zipping/Unzipping in Python
I am writing a simple batch script, and I need to delete one line from the file that gets downloaded by the script. What is the easiest way to do this using ONLY the command prompt? I have come across several various suggestions, but nothing to very simply delete one string that is constant across all files that are downloaded with the script.
This is a very platform dependant question. If you are using a *nix environment (Linux / Mac environment using bash / shell), you can accomplish this with sed
sed '/${regular_expression_that_matches_line_to_be_removed}/d' yourFile.txt > newFile.txt
This will generate a new file called newFile.txt that will contain the output. You can also do this in place (it modifies the file it's using as input), but I recommend against that because if you mess up your regex, you've lost your input.
If you're using a Windows environment (which I assume you are due to your batch-file tag), try looking at this Delete certain lines in a txt file via a batch file
As it appears there is no version of DOS (6.22 to WinME "DOS") or FreeDOS that allows you to take part of a text file and make it a variable, I'm going to just keep collecting the data I get in DOS mode into one very large file but I can't think of a way to get each asset and UUID and add them together in a third file... Here is what I get at the moment:
SMBIOS.TXT
~~~~~~~(usually 27 lines of stuff I don't need)~~~~~~~
Asset Number: ABC12345
~~~~~~~(usually 37 lines of stuff I don't need)~~~~~~~
UUID: ABCDEF12345678901234567890
~~~~~~~(usually 4 lines of stuff I don't need, complicated by a # symbol in there too)~~~~~~~
^Repeated many times
I need to add both the Asset Number and the UUID together in a CSV format so I was previously hoping (before I exhausted all attempts at doing for /F in DOS) just echoing the two variables I was creating as follows:
ECHO !Asset!,!UUID!>>Results.CSV
Which again works in Windows command prompt just not DOS, the script I'm using however only gets the first variable of each so I need to do them in order and keep repeating through the file in the manner?
Alternatively is there any other way I could use DOS to get the info I need out of the two text files on the fly? It's all running from a USB stick so I don't have any size constraints at least.
Aaron
You may be able to get it done with the DOS command line text stream editor EDLIN.
I am assuming this has to be done in DOS and Windows is not an option.
PDGREPPE is an MS DOS command line GREP search and replace utility.
Maybe you can find a DOS Text Editor with Macros
Lotus 123?
DBase?
Write an app in DOS BASIC.
I finally gave up after 120 hours of not finding a .NET or javascript/jquery plugin that can read and write to a COMPRESSED custom user frame (TXXX) in ID3v2 MP3 audio file.
UltraID3Lib: cannot read or write COMPRESSED Frames (last updated 2009, author Mitchell S. Honnert fell off face of the earth).
ID3Lib-sharp: cannot read or write COMPRESSED Frames (last updated 2012)
JavaScript-ID3-Reader: can return bytes but it's mostly the wrong bytes. Cannot write anything.
I cannot use the multitudes of Node.js or PHP scripts for my project so they are out of the question.
The only code I found that can read and write compress frames is Kid3.
http://sourceforge.net/projects/kid3/
However, it is written in C (I don't know that) and uses third party frameworks since it was not built in windows. The Command program required 13MB of support dlls, QMs, and whatevers.
I have no choice at his point but to try and use it's separate command program kid3-cli.exe
So here's my question:
Here is the way to read a TXXX frame using the program at the command prompt.
"71F3-15-FOO58A77" is the name of the TXXX frame and the "2" gets the text value it holds:
cd "C:\mp3folder"
select "test.mp3"
get "71F3-15-FOO58A77" "2"
export "clipboard" "CSV unquoted" "2"
QUESTION: HOW DO I use a Batch file to run these commands?
According to kid3 documents. It shows using -c as grouped commands. Windows cmd (or the program) on windows does know what -c is.
Example: I double click the batch file and it should:
start the program
sent the program (not cmd) the above 4 lines
each must be executed separately.
Sounds simple, but I can't get it to even execute one of the programs code after starting.
Any ideas? and and can someone write a ID3 tag program that can read and write COMPRESSED TXXX tags without using node.js, PHP or a server in Windows.
I will buy them a beer because I'm really a (cheap) designer by trade and a pert-time programmer only when I have too.
Here is a zip file of a COMPRESSED TXXX Frame in test.mp3 to test:
[http://robbiestewart.ca/test.zip][1]
Download Kid3 and use its windows GUI (kid3.exe) to view the custom user frame (TXXX).
Run the included kid3-cli.exe to do the same at the command prompt.
Try to do the same in a batch file.
According to the help file, you should be able to use the command
kid3-cli -c 'cd "C:\mp3folder"' -c 'select "test.mp3"' -c 'get "71F3-15-F0058A77" "2"' -c 'export "clipboard" "CSV unquoted" "2"'
I ran it on the file you provided and seven tabs followed by 0:00.00 were put on my clipboard, but the value of the TXXX field indicated by the GUI was output to my command prompt.
Description:
I am searching a very large server for files that is on a different server. right now I open command prompt and type
DIR [FILE NAME] /S/4
This returns the server location of the file with some other stuff that is not really needed.
Question:
I have a lot of files to search and one by one input into the above command could take forever. Is there a way I could input all of the names of all the files and only search once and the search results would only need to show file name and location?
First, I hope you don't mean DOS, but rather Windows cmd or batch.
You can certainly write a script that will run your DIR command once per file being sought.
But what you most likely want instead is to search once and print the path of each file found. For this you can use PowerShell's FindChildItem or the improved one posted here: http://windows-powershell-scripts.blogspot.in/2009/08/unix-linux-find-equivalent-in.html
It will be something like:
Find-ChildItem -Name "firstfile.txt|secondfile.txt|..."
Another approach is to install msys or cygwin or another Linux tools environment for Windows and use the Linux find command.