How to reverse the content of binary file? Unicorn Meta Zoo #1: Why another podcast? ...
Rolling Stones Sway guitar solo chord function
"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"
What is the term for a person whose job is to place products on shelves in stores?
How would this chord from "Rocket Man" be analyzed?
Would reducing the reference voltage of an ADC have any effect on accuracy?
How to keep bees out of canned beverages?
Is Diceware more secure than a long passphrase?
What *exactly* is electrical current, voltage, and resistance?
Is it OK if I do not take the receipt in Germany?
Trumpet valves, lengths, and pitch
What is the least dense liquid under normal conditions?
What was Apollo 13's "Little Jolt" after MECO?
Multiple options vs single option UI
Why didn't the Space Shuttle bounce back into space as many times as possible so as to lose a lot of kinetic energy up there?
Are there moral objections to a life motivated purely by money? How to sway a person from this lifestyle?
What is this word supposed to be?
Can I criticise the more senior developers around me for not writing clean code?
A Paper Record is What I Hamper
How can I wire a 9-position switch so that each position turns on one more LED than the one before?
Check if a string is entirely made of the same substring
What is the ongoing value of the Kanban board to the developers as opposed to management
c++ diamond problem - How to call base method only once
Align column where each cell has two decimals with siunitx
I preordered a game on my Xbox while on the home screen of my friend's account. Which of us owns the game?
How to reverse the content of binary file?
Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
2019 Community Moderator Election Results
Why I closed the “Why is Kali so hard” questionHow to use bash script to read binary file content?Why using cat on binary files messed up the terminal and how?How could i flip content of a binary file with bash commandsConvert binary mode to text mode and the reverse optionSplit binary data of fixed byte offset by byte position?Extract data between two matched patterns in a binary fileHow to view a binary file?How are binary files “binary”?Text file being identified as binarylinux aplication binary files location - how to find out?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
I was solving a challenge where I found a data file with no file extension. The file
command shows that it is a data file (application/octet-stream)
. The hd
command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.
binary
add a comment |
I was solving a challenge where I found a data file with no file extension. The file
command shows that it is a data file (application/octet-stream)
. The hd
command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.
binary
add a comment |
I was solving a challenge where I found a data file with no file extension. The file
command shows that it is a data file (application/octet-stream)
. The hd
command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.
binary
I was solving a challenge where I found a data file with no file extension. The file
command shows that it is a data file (application/octet-stream)
. The hd
command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.
binary
binary
edited 2 hours ago
Solomon Ucko
10916
10916
asked Jan 11 '18 at 17:44
Prvt_YadvPrvt_Yadv
3,35631531
3,35631531
add a comment |
add a comment |
5 Answers
5
active
oldest
votes
With xxd
(from vim
) and tac
(from GNU coreutils, also tail -r
on some systems):
< file.gnp xxd -p -c1 | tac | xxd -p -r > file.png
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
add a comment |
With perl
:
<file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png
-a
:awk
mode were records are split into fields
-0777 -p
: slurp-mode, file read into one$_
record, modified record printed afterwards.
-F
: empty field separator, so fields are individual bytes
$_=reverse@F
: output record is the concatenation of the list of fields (@F
) reversed.
add a comment |
In zsh
(the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):
zmodload zsh/mapfile
(LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)
LC_ALL=C
: characters are bytes
$mapfile[file.gnp]
: content offile.gnp
file
s::
: split the string into its byte constituents
Oa
: reverseO
rder ona
rray subscript that array
zsh
is not the only shell that can handle binary data.
– fpmurphy
Jan 12 '18 at 14:21
add a comment |
Here is one way of reversing a binary file using ksh93
. I have left the code "loose" to make it easier to understand.
#!/bin/ksh93
typeset -b byte
redirect 3< image.gpj || exit 1
eof=$(3<#((EOF)))
read -r -u 3 -N 1 byte
printf "%B" byte > image.jpg
3<#((CUR - 1))
while (( $(3<#) > 0 ))
do
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
3<#((CUR - 2))
done
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
redirect 3<&- || echo 'cannot close FD 3'
exit 0
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the firstread
above should read nothing as it's done at the end of the file.
– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it understrace
andksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15
– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Trystrace
on the script to see what I mean.ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.
– Stéphane Chazelas
Jan 13 '18 at 9:23
add a comment |
I tried the following:
tac -rs '.' input.gnp > output.png
The idea is to force 'tac' using any character as separator.
I tried that on a binary file and it seemed to work but any confirmation would be appreciated.
Main advantage is that it does not load file into memory.
Doesn't work for me (here with GNUtac
8.28) when the input contains newline characters.printf '1n2' | tac -rs . | od -vAn -tc
outputsn 2 1
instead of2 n 1
. You'd also needLC_ALL=C
or.
could match multi-byte characters.
– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.
– Stéphane Chazelas
Nov 15 '18 at 20:25
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f416401%2fhow-to-reverse-the-content-of-binary-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
With xxd
(from vim
) and tac
(from GNU coreutils, also tail -r
on some systems):
< file.gnp xxd -p -c1 | tac | xxd -p -r > file.png
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
add a comment |
With xxd
(from vim
) and tac
(from GNU coreutils, also tail -r
on some systems):
< file.gnp xxd -p -c1 | tac | xxd -p -r > file.png
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
add a comment |
With xxd
(from vim
) and tac
(from GNU coreutils, also tail -r
on some systems):
< file.gnp xxd -p -c1 | tac | xxd -p -r > file.png
With xxd
(from vim
) and tac
(from GNU coreutils, also tail -r
on some systems):
< file.gnp xxd -p -c1 | tac | xxd -p -r > file.png
edited Jan 12 '18 at 6:53
answered Jan 11 '18 at 21:41
Stéphane ChazelasStéphane Chazelas
315k57599957
315k57599957
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
add a comment |
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(
– Iulian Onofrei
May 12 '18 at 17:26
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
This is not a solution because it will mirror all the file.
– Philippe Delteil
Nov 21 '18 at 3:33
add a comment |
With perl
:
<file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png
-a
:awk
mode were records are split into fields
-0777 -p
: slurp-mode, file read into one$_
record, modified record printed afterwards.
-F
: empty field separator, so fields are individual bytes
$_=reverse@F
: output record is the concatenation of the list of fields (@F
) reversed.
add a comment |
With perl
:
<file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png
-a
:awk
mode were records are split into fields
-0777 -p
: slurp-mode, file read into one$_
record, modified record printed afterwards.
-F
: empty field separator, so fields are individual bytes
$_=reverse@F
: output record is the concatenation of the list of fields (@F
) reversed.
add a comment |
With perl
:
<file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png
-a
:awk
mode were records are split into fields
-0777 -p
: slurp-mode, file read into one$_
record, modified record printed afterwards.
-F
: empty field separator, so fields are individual bytes
$_=reverse@F
: output record is the concatenation of the list of fields (@F
) reversed.
With perl
:
<file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png
-a
:awk
mode were records are split into fields
-0777 -p
: slurp-mode, file read into one$_
record, modified record printed afterwards.
-F
: empty field separator, so fields are individual bytes
$_=reverse@F
: output record is the concatenation of the list of fields (@F
) reversed.
answered Jan 11 '18 at 21:46
Stéphane ChazelasStéphane Chazelas
315k57599957
315k57599957
add a comment |
add a comment |
In zsh
(the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):
zmodload zsh/mapfile
(LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)
LC_ALL=C
: characters are bytes
$mapfile[file.gnp]
: content offile.gnp
file
s::
: split the string into its byte constituents
Oa
: reverseO
rder ona
rray subscript that array
zsh
is not the only shell that can handle binary data.
– fpmurphy
Jan 12 '18 at 14:21
add a comment |
In zsh
(the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):
zmodload zsh/mapfile
(LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)
LC_ALL=C
: characters are bytes
$mapfile[file.gnp]
: content offile.gnp
file
s::
: split the string into its byte constituents
Oa
: reverseO
rder ona
rray subscript that array
zsh
is not the only shell that can handle binary data.
– fpmurphy
Jan 12 '18 at 14:21
add a comment |
In zsh
(the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):
zmodload zsh/mapfile
(LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)
LC_ALL=C
: characters are bytes
$mapfile[file.gnp]
: content offile.gnp
file
s::
: split the string into its byte constituents
Oa
: reverseO
rder ona
rray subscript that array
In zsh
(the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):
zmodload zsh/mapfile
(LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)
LC_ALL=C
: characters are bytes
$mapfile[file.gnp]
: content offile.gnp
file
s::
: split the string into its byte constituents
Oa
: reverseO
rder ona
rray subscript that array
edited Jan 12 '18 at 14:29
answered Jan 11 '18 at 21:36
Stéphane ChazelasStéphane Chazelas
315k57599957
315k57599957
zsh
is not the only shell that can handle binary data.
– fpmurphy
Jan 12 '18 at 14:21
add a comment |
zsh
is not the only shell that can handle binary data.
– fpmurphy
Jan 12 '18 at 14:21
zsh
is not the only shell that can handle binary data.– fpmurphy
Jan 12 '18 at 14:21
zsh
is not the only shell that can handle binary data.– fpmurphy
Jan 12 '18 at 14:21
add a comment |
Here is one way of reversing a binary file using ksh93
. I have left the code "loose" to make it easier to understand.
#!/bin/ksh93
typeset -b byte
redirect 3< image.gpj || exit 1
eof=$(3<#((EOF)))
read -r -u 3 -N 1 byte
printf "%B" byte > image.jpg
3<#((CUR - 1))
while (( $(3<#) > 0 ))
do
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
3<#((CUR - 2))
done
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
redirect 3<&- || echo 'cannot close FD 3'
exit 0
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the firstread
above should read nothing as it's done at the end of the file.
– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it understrace
andksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15
– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Trystrace
on the script to see what I mean.ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.
– Stéphane Chazelas
Jan 13 '18 at 9:23
add a comment |
Here is one way of reversing a binary file using ksh93
. I have left the code "loose" to make it easier to understand.
#!/bin/ksh93
typeset -b byte
redirect 3< image.gpj || exit 1
eof=$(3<#((EOF)))
read -r -u 3 -N 1 byte
printf "%B" byte > image.jpg
3<#((CUR - 1))
while (( $(3<#) > 0 ))
do
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
3<#((CUR - 2))
done
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
redirect 3<&- || echo 'cannot close FD 3'
exit 0
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the firstread
above should read nothing as it's done at the end of the file.
– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it understrace
andksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15
– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Trystrace
on the script to see what I mean.ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.
– Stéphane Chazelas
Jan 13 '18 at 9:23
add a comment |
Here is one way of reversing a binary file using ksh93
. I have left the code "loose" to make it easier to understand.
#!/bin/ksh93
typeset -b byte
redirect 3< image.gpj || exit 1
eof=$(3<#((EOF)))
read -r -u 3 -N 1 byte
printf "%B" byte > image.jpg
3<#((CUR - 1))
while (( $(3<#) > 0 ))
do
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
3<#((CUR - 2))
done
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
redirect 3<&- || echo 'cannot close FD 3'
exit 0
Here is one way of reversing a binary file using ksh93
. I have left the code "loose" to make it easier to understand.
#!/bin/ksh93
typeset -b byte
redirect 3< image.gpj || exit 1
eof=$(3<#((EOF)))
read -r -u 3 -N 1 byte
printf "%B" byte > image.jpg
3<#((CUR - 1))
while (( $(3<#) > 0 ))
do
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
3<#((CUR - 2))
done
read -r -u 3 -N 1 byte
printf "%B" byte >> image.jpg
redirect 3<&- || echo 'cannot close FD 3'
exit 0
edited Jan 12 '18 at 14:23
answered Jan 12 '18 at 14:15
fpmurphyfpmurphy
2,476916
2,476916
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the firstread
above should read nothing as it's done at the end of the file.
– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it understrace
andksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15
– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Trystrace
on the script to see what I mean.ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.
– Stéphane Chazelas
Jan 13 '18 at 9:23
add a comment |
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the firstread
above should read nothing as it's done at the end of the file.
– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it understrace
andksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15
– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Trystrace
on the script to see what I mean.ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.
– Stéphane Chazelas
Jan 13 '18 at 9:23
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s
– Stéphane Chazelas
Jan 12 '18 at 15:51
Note that the first
read
above should read nothing as it's done at the end of the file.– Stéphane Chazelas
Jan 12 '18 at 15:58
Note that the first
read
above should read nothing as it's done at the end of the file.– Stéphane Chazelas
Jan 12 '18 at 15:58
Trying to understand why it was so slow, I tried running it under
strace
and ksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15– Stéphane Chazelas
Jan 12 '18 at 16:00
Trying to understand why it was so slow, I tried running it under
strace
and ksh93
seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15– Stéphane Chazelas
Jan 12 '18 at 16:00
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
@StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.
– fpmurphy
Jan 13 '18 at 5:49
Try
strace
on the script to see what I mean. ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.– Stéphane Chazelas
Jan 13 '18 at 9:23
Try
strace
on the script to see what I mean. ksh93
reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.– Stéphane Chazelas
Jan 13 '18 at 9:23
add a comment |
I tried the following:
tac -rs '.' input.gnp > output.png
The idea is to force 'tac' using any character as separator.
I tried that on a binary file and it seemed to work but any confirmation would be appreciated.
Main advantage is that it does not load file into memory.
Doesn't work for me (here with GNUtac
8.28) when the input contains newline characters.printf '1n2' | tac -rs . | od -vAn -tc
outputsn 2 1
instead of2 n 1
. You'd also needLC_ALL=C
or.
could match multi-byte characters.
– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.
– Stéphane Chazelas
Nov 15 '18 at 20:25
add a comment |
I tried the following:
tac -rs '.' input.gnp > output.png
The idea is to force 'tac' using any character as separator.
I tried that on a binary file and it seemed to work but any confirmation would be appreciated.
Main advantage is that it does not load file into memory.
Doesn't work for me (here with GNUtac
8.28) when the input contains newline characters.printf '1n2' | tac -rs . | od -vAn -tc
outputsn 2 1
instead of2 n 1
. You'd also needLC_ALL=C
or.
could match multi-byte characters.
– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.
– Stéphane Chazelas
Nov 15 '18 at 20:25
add a comment |
I tried the following:
tac -rs '.' input.gnp > output.png
The idea is to force 'tac' using any character as separator.
I tried that on a binary file and it seemed to work but any confirmation would be appreciated.
Main advantage is that it does not load file into memory.
I tried the following:
tac -rs '.' input.gnp > output.png
The idea is to force 'tac' using any character as separator.
I tried that on a binary file and it seemed to work but any confirmation would be appreciated.
Main advantage is that it does not load file into memory.
answered Apr 16 '18 at 16:00
BouteilleBouteille
1
1
Doesn't work for me (here with GNUtac
8.28) when the input contains newline characters.printf '1n2' | tac -rs . | od -vAn -tc
outputsn 2 1
instead of2 n 1
. You'd also needLC_ALL=C
or.
could match multi-byte characters.
– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.
– Stéphane Chazelas
Nov 15 '18 at 20:25
add a comment |
Doesn't work for me (here with GNUtac
8.28) when the input contains newline characters.printf '1n2' | tac -rs . | od -vAn -tc
outputsn 2 1
instead of2 n 1
. You'd also needLC_ALL=C
or.
could match multi-byte characters.
– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.
– Stéphane Chazelas
Nov 15 '18 at 20:25
Doesn't work for me (here with GNU
tac
8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc
outputs n 2 1
instead of 2 n 1
. You'd also need LC_ALL=C
or .
could match multi-byte characters.– Stéphane Chazelas
Nov 15 '18 at 20:23
Doesn't work for me (here with GNU
tac
8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc
outputs n 2 1
instead of 2 n 1
. You'd also need LC_ALL=C
or .
could match multi-byte characters.– Stéphane Chazelas
Nov 15 '18 at 20:23
LC_ALL=C tac -rs $'.\|n'
seems to work though.– Stéphane Chazelas
Nov 15 '18 at 20:25
LC_ALL=C tac -rs $'.\|n'
seems to work though.– Stéphane Chazelas
Nov 15 '18 at 20:25
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f416401%2fhow-to-reverse-the-content-of-binary-file%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown