I have a python script that runs a C program :
cmd = ["/Users/stordd/Desktop/StageI2M/C/forestenostre/grezza_foresta", "-w", "/Users/stordd/Desktop/StageI2M/Leiden/text_file/USA.txt", "-m", "5", "-e", "-0"]
# Open/Create the output file
outFile = open("/Users/stordd/Desktop/StageI2M/Leiden/text_file/Output.txt", 'wb')
result = subprocess.Popen(cmd, stdout=subprocess.PIPE)
out = result.stdout.read()
outFile.write(out)
outFile.close()
It takes a text file as input and create a text file as output and I want to make a loop that would create a text file at each iteration without replacing the previous one. How can I do that ?
In the open function call add the "a" for append
outFile = open("/Users/stordd/Desktop/StageI2M/Leiden/text_file/Output.txt", 'ab')
https://docs.python.org/3/library/functions.html#open
Related
I was trying to create a Perl script to rename the files (hundreds of files with different names), but I have not had any success. I first need to find the unique file number and then rename it to something more human readable. Since file names are not sequential, it makes it difficult.
Examples of files names: The number of importance is after que sequence
# vv-- this number
lane8-s244-index--ATTACTCG-TATAGCCT-01_S244_L008_R1_001.fastq
lane8-s245-index--ATTACTCG-ATAGAGGC-02_S245_L008_R1_001.fastq
lane8-s246-index--TCCGGAGA-TATAGCCT-09_S246_L008_R1_001.fastq
lane8-s247-index--TCCGGAGA-ATAGAGGC-10_S247_L008_R1_001.fastq
lane8-s248-index--TCCGGAGA-CCTATCCT-11_S248_L008_R1_001.fastq
lane8-s249-index--TCCGGAGA-GGCTCTGA-12_S249_L008_R1_001.fastq
lane8-s250-index--TCCGGAGA-AGGCGAAG-13_S250_L008_R1_001.fastq
lane8-s251-index--TCCGGAGA-TAATCTTA-14_S251_L008_R1_001.fastq
lane7-s0007-index--ATTACTCG-TATAGCCT-193_S7_L007_R1_001.fastq
lane7-s0008-index--ATTACTCG-ATAGAGGC-105_S8_L007_R1_001.fastq
lane7-s0009-index--ATTACTCG-CCTATCCT-195_S9_L007_R1_001.fastq
lane7-s0010-index--ATTACTCG-GGCTCTGA-106_S10_L007_R1_001.fastq
lane7-s0011-index--ATTACTCG-AGGCGAAG-197_S11_L007_R1_001.fastq
lane7-s0096-index--AGCGATAG-CAGGACGT-287_S96_L007_R1_001.fastq
I have created a file called RENAMING_parse_data.sh that reference RENAMING_parse_data.pl
So in theory the idea is that it is parsing the data to find the sample # that is in the middle of the name, and taking that unique ID and renaming it. But I don't think it's even going into the IF loop.
Any ideas?
HERE IS THE .sh file that calls the perl scipt
#!/bin/bash
#first part is the program
#second is the directory path
#third and fourth times are the names of the output files
#./parse_data.pl /ACTF/Course/PATHTDIRECTORY Tabsummary.txt Strucsummary.txt
#WHERE ./parse_data.pl =name of the program
#WHERE /ACTF/Course/PATHTODIRECTORY = directory path were your field are saved AND is referred to as $dir_in = $ARGV[0] in the perl script;
#new files you recreating with the extracted data AND is refered to as $dir_in = $ARGV[1];
./RENAMING_parse_data.pl ./Test/ FishList.txt
HERE IS THE PERL SCRIP:
#!/usr/bin/perl
print (":)\n");
#Proesessing files in a directory
$dir_in = $ARGV[0];
$indv_list = $ARGV[1];
#open directory to acess those files, the folder where you have the files
opendir(DIR, $dir_in) || die ("Cannot open $dir_in");
#files = readdir(DIR);
#set all variables = 0 to void chaos
$j=0;
#open output header line for output file and print header line for tab delimited file
open(OUTFILETAB, ">", $indv_list);
print(OUTFILETAB "\t Fish ID", "\t");
#open each file
foreach (#files){
#re start all arrays to void chaos
print("in loop [$j]");
#acc_ID=();
#find FISH name
#EXAMPLE FISH NAMES: (lenth of fishname varies)
#lane8-s251-index--TCCGGAGA-TAATCTTA-14_S251_L008_R1_001.fastq.gz
#lane7-s0096-index--AGCGATAG-CAGGACGT-287_S96_L007_R1_001.final.fastq
#NOTE: what is in btween () is the ID that is printed NOTE that value can change from 2 -3 depending on Sample #
#Trials:
#lane[0-9]{1}-[a-z]{1}[0-9]{4}-index--[A-Z]{8}[A-Z]{8}-([0-9]{3})[a-z]{1}[0-9]{2}_[A-Z]{1}[0-9]{3}_[a-z]{1}[0-9]{1}_[0-9]{3}.fastq
#lane[0-9]{1}-[a-z]{1}[0-9]{4}-index--[A-Z]{8}[A-Z]{8}-([0-9]{3})*.fastq
#lane*([0-9]{3})*.fastq
#lane.*-([0-9]{2})_.*.fastq
#lane.*-([0-9]{2})_*.fastq
#lane[0-9]{1}-[a-z]{1}[0-9]{3}-index--[A-Z]{8}[A-Z]{8}-([0-9]{2})_[A-Z]{1}[0-9]{3}_L008_R1_001.fastq
$string_FISH = #files;
if ($string_FISH =~ /^lane[0-9]{1}-[a-z]{1}[0-9]{3}-index--[A-Z]{8}[A-Z]{8}-([0-9]{2})_[A-Z]{1}[0-9]{3}_L008_R1_001.fastq/){
$FISH_ID =$1;
#acc_ID[$j] = $FISH_ID;
#print ("FISH. = |$FISH_ID[$j]| \n");
rename($string_FISH, "FISH. = |$FISH_ID[$j]|");
#print ($acc_ID[$j], "\n");
print(OUTFILETAB "FISH. = |$FISH_ID[$j]| \n");
}
$j= $j+1;
}
IDEAL END RESULT
So in the end I would like it to take the file name, find the unique identifier and rename it
from :
lane8-s244-index--ATTACTCG-TATAGCCT-01_S244_L008_R1_001.fastq
lane7-s0007-index--ATTACTCG-TATAGCCT-193_S7_L007_R1_001.fastq
to:
Fish.01.fastq
Fish.193.fastq
Any Ideas or suggestion on hot to fix this or If it need to change completely are greatly appreciated.
At the core of a Perl solution, you could use
s/^.*-(\d+)_[^-]+(?=\.fastq\z)/Fish.$1/sa
For example,
$ ls -1 *.fastq
lane8-s244-index--ATTACTCG-TATAGCCT-01_S244_L008_R1_001.fastq
lane8-s245-index--ATTACTCG-ATAGAGGC-02_S245_L008_R1_001.fastq
lane8-s246-index--TCCGGAGA-TATAGCCT-09_S246_L008_R1_001.fastq
lane8-s247-index--TCCGGAGA-ATAGAGGC-10_S247_L008_R1_001.fastq
lane8-s248-index--TCCGGAGA-CCTATCCT-11_S248_L008_R1_001.fastq
lane8-s249-index--TCCGGAGA-GGCTCTGA-12_S249_L008_R1_001.fastq
$ rename 's/^.*-(\d+)_[^-]+(?=\.fastq\z)/Fish.$1/sa' *.fastq
$ ls -1 *.fastq
Fish.01.fastq
Fish.02.fastq
Fish.09.fastq
Fish.10.fastq
Fish.11.fastq
Fish.12.fastq
(There are two similar tools named rename. This one is also known as prename.)
It's pretty simple to implement yourself:
#!/usr/bin/perl
use strict;
use warnings;
my $errors = 0;
for (#ARGV) {
my $old = $_;
s/^.*-(\d+)_[^-]+(?=\.fastq\z)/Fish.$1/sa;
my $new = $_;
next if $new eq $old;
if ( -e $new ) {
warn( "Can't rename \"$old\" to \"$new\": Already exists\n" );
++$errors;
}
elsif ( !rename( $old, $new ) ) {
warn( "Can't rename \"$old\" to \"$new\": $!\n" );
++$errors;
}
}
exit( !!$errors );
Provide the files to rename as arguments (e.g. using *.fastq from the shell).
$ ls -1 *.fastq
lane8-s244-index--ATTACTCG-TATAGCCT-01_S244_L008_R1_001.fastq
lane8-s245-index--ATTACTCG-ATAGAGGC-02_S245_L008_R1_001.fastq
lane8-s246-index--TCCGGAGA-TATAGCCT-09_S246_L008_R1_001.fastq
lane8-s247-index--TCCGGAGA-ATAGAGGC-10_S247_L008_R1_001.fastq
lane8-s248-index--TCCGGAGA-CCTATCCT-11_S248_L008_R1_001.fastq
lane8-s249-index--TCCGGAGA-GGCTCTGA-12_S249_L008_R1_001.fastq
$ ./a *.fastq
$ ls -1 *.fastq
Fish.01.fastq
Fish.02.fastq
Fish.09.fastq
Fish.10.fastq
Fish.11.fastq
Fish.12.fastq
The existence check (-e) is to prevent accidentally renaming a bunch of files to the same name and therefore losing all but one of them.
The above is an cleaned up version of an one-liner pattern I often use.
dir /b ... | perl -nle"$o=$_; s/.../.../; $n=$_; rename$o,$n if!-e$n"
Adapted to sh:
\ls ... | perl -nle'$o=$_; s/.../.../; $n=$_; rename$o,$n if!-e$n'
This question already has an answer here:
In Perl, how do I change, delete, or insert a line in a file, or append to the beginning of a file?
(1 answer)
Closed 1 year ago.
y.txt -> file to be edited
x.txt -> file that needs to be copied and written in y.txt
y.txt has
c...
w...
// search for this comment and copy the entire x.txt file here
// search end comment when you need to delete lines between these comments
g...
How do I do this in Perl? Also how do I search for both the comments and delete the lines between them?
So y.txt looks like this after insertion
c...
w...
// search for this comment and copy the entire x.txt file here
Entire x.txt file here
// search end comment when you need to delete lines between these comments
g...
The file to be inserted is about 20 lines in total. I create this file as well.
Here is one approach:
use strict;
use warnings;
my $xfn = 'x.txt';
open ( my $fh, '<', $xfn ) or die "Could not open file '$xfn': $!";
my $xstr = my $str = do { local $/; <$fh> };
close $fh;
#ARGV = qw(y.txt);
$^I = '.bak'; # Backup file extension
my $delete = 0;
while(<<>>) {
if (m{// search end comment}) {
$delete = 0;
}
next if $delete;
print;
if (m{// search for this comment}) {
$delete = 1;
print $xstr;
}
}
i want to open up a pre-existed file and want to add a string inside the file one line before it sees the word 'exit' inside the file. the word 'exit' will always be the last line inside the file, so we can also see this as " add the string one line above the last line" problem. in other words, I want to append this string inside the file. here is example
Example.tcl (before)
AAAAAAA
BBBBBBB
CCCCCC
exit
Example.tcl (after)
AAAAAAA
BBBBBBB
CCCCCC
new_word_string
exit
Any suggestions are most welcome.
Working code:
Open the file for reading, and also open a temporary file:
set f1 [open $thefile]
set f2 [file tempfile]
Read one line at a time until all lines have been read. Look at the line. If it is the string "exit", print the new string to the temporary file. The write the line you read to the temporary file.
while {[set line [chan gets $f1]] ne {}} {
if {$line eq "exit"} {
chan puts $f2 $thestring
}
chan puts $f2 $line
}
Close the file and reopen it for reading.
chan close $f1
set f1 [open $thefile w]
Rewind the temporary file to the start position.
chan seek $f2 0
Read the entire contents of the temporary file and print them to the file.
chan puts -nonewline $f1 [chan read -nonewline $f2]
Close both files.
chan close $f1
chan close $f2
And we're done.
You could use a string buffer instead of a temporary file with minimal changes, to wit:
set f [open $thefile]
set tempstr {}
while {[set line [chan gets $f]] ne {}} {
if {$line eq "exit"} {
append tempstr $thestring\n
}
append tempstr $line\n
}
chan close $f
set f [open $thefile w]
chan puts -nonewline $f $tempstr
chan close $f
Documentation: append, chan, if, open, set, while
You could farm the work out to an external command (Tcl was written as a glue language after all):
% exec cat example.tcl
AAAAAAA
BBBBBBB
CCCCCC
exit
% set new_line "this is the new line inserted before exit"
this is the new line inserted before exit
% exec sed -i "\$i$new_line" example.tcl
% exec cat example.tcl
AAAAAAA
BBBBBBB
CCCCCC
this is the new line inserted before exit
exit
I need to create a dynamic variable name for some arrays and fill it with multi-line text.
What I actually have is this :
#!/bin/bash
IFS=$'\n'
# Set an array with 1 item
ARRAY=("Item1")
# Get a description for the last item from a simple text file
DESCRIPTION=("$(cat test1.txt)")
# Get the number of items in the array
ARRAY_ITEMS_COUNT=${#ARRAY[#]}
# Create a variable name containing the number of items in the array as identifier
# and fill it with the description
eval ARRAY_ITEM${ARRAY_ITEMS_COUNT}_DESCRIPTION="(\"${DESCRIPTION[#]}\")"
# Display some results
echo "ARRAY_ITEM1_DESCRIPTION[#] = \"${ARRAY_ITEM1_DESCRIPTION[#]}\""
echo "ARRAY_ITEM1_DESCRIPTION[0] = \"${ARRAY_ITEM1_DESCRIPTION[0]}\""
echo "ARRAY_ITEM1_DESCRIPTION[1] = \"${ARRAY_ITEM1_DESCRIPTION[1]}\""
echo
# Same as above with a different text file
ARRAY=("Item1" "Item2")
DESCRIPTION=("$(cat test2.txt)")
ARRAY_ITEMS_COUNT=${#ARRAY[#]}
# Get an error here due to the ' character used in the text file
eval ARRAY_ITEM${ARRAY_ITEMS_COUNT}_DESCRIPTION="(\"${DESCRIPTION[#]}\")"
echo "ARRAY_ITEM2_DESCRIPTION[#] = \"${ARRAY_ITEM2_DESCRIPTION[#]}\""
echo "ARRAY_ITEM2_DESCRIPTION[0] = \"${ARRAY_ITEM2_DESCRIPTION[0]}\""
echo "ARRAY_ITEM2_DESCRIPTION[1] = \"${ARRAY_ITEM2_DESCRIPTION[1]}\""
The files "test1.txt" and "test2.txt" are as follow :
test1.txt
Simple text file with multi-lines used as
a test without special characters inside.
test2.txt
Simple text file with multi-lines used as
a test with single ' and double " quotes.
Expected result :
ARRAY_ITEM1_DESCRIPTION[#] = "Simple text file with multi-lines used as
a test without special characters inside."
ARRAY_ITEM1_DESCRIPTION[0] = "Simple text file with multi-lines used as"
ARRAY_ITEM1_DESCRIPTION[1] = "a test without special characters inside."
ARRAY_ITEM2_DESCRIPTION[#] = "Simple text file with multi-lines used as
a test with single ' and double " quotes."
ARRAY_ITEM2_DESCRIPTION[0] = "Simple text file with multi-lines used as"
ARRAY_ITEM2_DESCRIPTION[1] = "a test with single ' and double " quotes."
Current result :
ARRAY_ITEM1_DESCRIPTION[#] = "Simple text file with multi-lines used as
a test without special characters inside."
ARRAY_ITEM1_DESCRIPTION[0] = "Simple text file with multi-lines used as
a test without special characters inside."
ARRAY_ITEM1_DESCRIPTION[1] = ""
./test.sh: eval: line 28: unexpected EOF while looking for matching `"'
./test.sh: eval: line 29: syntax error: unexpected end of file
ARRAY_ITEM2_DESCRIPTION[#] = ""
ARRAY_ITEM2_DESCRIPTION[0] = ""
ARRAY_ITEM2_DESCRIPTION[1] = ""
I tried a lot of things but it never gives me what is expected, so can someone help me solve the 2 issues I have there please :
How to get proper array containing each line of the text file on each index
How to make it work even with quotes characters in the texts
EDIT : Working solution (bash version > 4) is :
#!/bin/bash
# Set an array with 1 item
ARRAY=("Item1")
# Get the number of items in the array
ARRAY_ITEMS_COUNT=${#ARRAY[#]}
# Create a variable name containing the number of items in the array as identifier
# and fill it with the description
readarray -t "ARRAY_ITEM${ARRAY_ITEMS_COUNT}_DESCRIPTION" < test1.txt
# Display some results
echo "ARRAY_ITEM1_DESCRIPTION[#] = \"${ARRAY_ITEM1_DESCRIPTION[#]}\""
echo "ARRAY_ITEM1_DESCRIPTION[0] = \"${ARRAY_ITEM1_DESCRIPTION[0]}\""
echo "ARRAY_ITEM1_DESCRIPTION[1] = \"${ARRAY_ITEM1_DESCRIPTION[1]}\""
echo
# Same as above with a different text file
ARRAY=("Item1" "Item2")
ARRAY_ITEMS_COUNT=${#ARRAY[#]}
# Get an error here due to the ' character used in the text file
readarray -t ARRAY_ITEM${ARRAY_ITEMS_COUNT}_DESCRIPTION < test2.txt
echo "ARRAY_ITEM2_DESCRIPTION[#] = \"${ARRAY_ITEM2_DESCRIPTION[#]}\""
echo "ARRAY_ITEM2_DESCRIPTION[0] = \"${ARRAY_ITEM2_DESCRIPTION[0]}\""
echo "ARRAY_ITEM2_DESCRIPTION[1] = \"${ARRAY_ITEM2_DESCRIPTION[1]}\""
Thanks for your help, have a nice day.
Slander
When you call cat in an array assignment you shouldn't quote it if you want the file to be read line by line. Because if you do so the contents of the file will be handled as one string/one line. So it won't get read line by line. Just try:
DESCRIPTION=($(cat test1.txt))
Also if you are using Bash version 4 you could use bash builtin command readarray to generate an array:
readarray -t DESCRIPTION < "test1.txt"
For Bash version < 4 this could be an alternative to cat and readarray:
IFS=$'\n' read -d -r -a DESCRIPTION < "test1.txt"
I have decided to have an accompanying .ini file with an executable so that I do not have to hard code items that appear in a drop down menu. I have created the .ini file and using the IniReadSection function I have been able to look through the section and output each Key=Value pair within that section.
How would I go about placing the value's only into a string array? I am writing this utility using AutoIT.
I made this ini file:
[JHamill]
key1=value1
key2=value2
key3=value3
I took a bit of code from IniReadSection example and modified this to be able to use it for a drop down menu.
$var = IniReadSection("test.ini", "JHamill")
$str = ""
For $i = 1 To $var[0][0]
$str &= $var[$i][1] & "|"
Next
$str = StringTrimRight($str, 1)
GUICreate("JHamill GUI combo")
GUICtrlCreateCombo("", 10, 10)
GUICtrlSetData(-1, $str)
GUISetState()
While 1
$msg = GUIGetMsg()
If $msg = -3 Then ExitLoop
WEnd
Here you see you don't have to make a new array to make it work. But since you asked, here is the same thing by making a new array first, copying only the value elements, and then using that array to fill the combo box:
#include <Array.au3>
$var = IniReadSection("test.ini", "JHamill")
Local $arr[$var[0][0]]
For $i = 1 To $var[0][0]
$arr[$i-1] = $var[$i][1]
Next
_ArrayDisplay($arr)