How to work with block in loop by using sed? - loops

Question

I have file with some blocks, like this:
<start> test var=3333
<g>test=000000000000 tst <s>
<end>
...
<start> var=564735628
<title>somethink<\title>
<end>
...
And I need to get block between and sections in a loop.
And then I need to get some simbols in the current block.
I try to do like this:
for block in $(cat $file | sed -n '/<start>/,/<end>/p;'); do
echo $block
done
Result is:
<start>
instead
<start> test 1
<g>test=000000000000 tst <s>
<end>
How can I get the entire block for further processing?
Ok, I try to explain
Source is
<start> test var=3333
<g>test=000000000000 tst <s>
<end>
Result of yours code is not a block. It is just a sting.
The string is <end>t> test var=3333tst <s>
As can you see it is overlaping strings of the block on each other.

Answer 1

One sugession, do not used sed here.
Use languages like perl or python which gives modules for parsing HTML and XML.

Answer 2

You could do something like:
block=""
cat $file | sed -n '/<start>/,/<end>/p;' | while read -r line; do
if [ -z "$block" ]; then
block="$line"
else
block=$(printf "%s\\n%s" "$block" "$line")
fi
if printf "%s\\n" "$line" | grep "<end>" > /dev/null; then
echo "$block"
block=""
fi
done
As choroba said in his answer, your for loop will use the IFS variable to split sed's output into separate fields, and the block variable will contain only a single field. (Ie., block will contain <start>, then test, then var=3333, and so on).
A solution is to force it read line by line, by piping the output of sed into the loop command, and read the line using the read command. The -r flag for the read command forces it not to interpret the backslash as an escaping character. Now we have a variable $line with our line, but not the block. To get the block, simply concatenate the lines together until we find the <end> string.
If the $block variable is empty, we can simply assign the $line to it. Otherwise, we use the printf command to generate a new string containing the previous value of $block concatanated with a newline character and the contents of $line. This newline character prevents that the block will become a single line.
To test if we found the last line, we can print the current value of the block and see if grep finds it. I used printf because it's safer then echo when the string we want to print starts with a variable (we can't guarantee that the variable doesn't start with a hyphen, which echo could interpret as an option). We must also remember to clear the block variable when we actually find a block, in order to prepare it for the next block.

Answer 3

Word splitting is applied to the output of your sed command. You can set IFS to an empty value to prevent word splitting on the sed output, but it will make the whole output of sed into one "block". I would rather switch to a more powerful language like Perl.

Answer 4

This might work for you (GNU sed and bash):
OIFS=$IFS; IFS=$'\n'; block=($(sed '/<start>/,/<end>/!d' file)); IFS=$OIFS
for x in "${!block[#]}"; do echo "${block[x]}"; done
Slurp the sed command output into an array block and loop through the array.

Answer 5

By changing IFS and inserting a delimiter character in between your blocks, you can iterate through each block.
For example, use : as the delimiter
OLDIFS=$IFS; IFS=':'
blocks=$(sed -n '/start/,/end/ {/start/ s/^/:/; p}' file)
for block in ${blocks#:}; do
echo "This is block $((count++))"
echo "$block"
done
IFS=$OLDIFS
Note:
Blocks are 'separated' by inserting : before <start> and setting IFS to :
${blocks#:} removes the first :, otherwise :block1:block2... is interpreted as emptyblock:block1:block2..., i.e. the loop iterates over the non-existent first block (which is empty and exists due to how : is placed)
Alternatively, : can be placed behind <end> but then the last line of the block would become <end>:\n so there would be an extra newline before the start of next block.