How to reverse the content of binary file? Unicorn Meta Zoo #1: Why another podcast? ...

Rolling Stones Sway guitar solo chord function

"My boss was furious with me and I have been fired" vs. "My boss was furious with me and I was fired"

What is the term for a person whose job is to place products on shelves in stores?

How would this chord from "Rocket Man" be analyzed?

Would reducing the reference voltage of an ADC have any effect on accuracy?

How to keep bees out of canned beverages?

Is Diceware more secure than a long passphrase?

What *exactly* is electrical current, voltage, and resistance?

Is it OK if I do not take the receipt in Germany?

Trumpet valves, lengths, and pitch

What is the least dense liquid under normal conditions?

What was Apollo 13's "Little Jolt" after MECO?

Multiple options vs single option UI

Why didn't the Space Shuttle bounce back into space as many times as possible so as to lose a lot of kinetic energy up there?

Are there moral objections to a life motivated purely by money? How to sway a person from this lifestyle?

What is this word supposed to be?

Can I criticise the more senior developers around me for not writing clean code?

A Paper Record is What I Hamper

How can I wire a 9-position switch so that each position turns on one more LED than the one before?

Check if a string is entirely made of the same substring

What is the ongoing value of the Kanban board to the developers as opposed to management

c++ diamond problem - How to call base method only once

Align column where each cell has two decimals with siunitx

I preordered a game on my Xbox while on the home screen of my friend's account. Which of us owns the game?



How to reverse the content of binary file?



Unicorn Meta Zoo #1: Why another podcast?
Announcing the arrival of Valued Associate #679: Cesar Manara
2019 Community Moderator Election Results
Why I closed the “Why is Kali so hard” questionHow to use bash script to read binary file content?Why using cat on binary files messed up the terminal and how?How could i flip content of a binary file with bash commandsConvert binary mode to text mode and the reverse optionSplit binary data of fixed byte offset by byte position?Extract data between two matched patterns in a binary fileHow to view a binary file?How are binary files “binary”?Text file being identified as binarylinux aplication binary files location - how to find out?





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







10















I was solving a challenge where I found a data file with no file extension. The file command shows that it is a data file (application/octet-stream). The hd command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.










share|improve this question































    10















    I was solving a challenge where I found a data file with no file extension. The file command shows that it is a data file (application/octet-stream). The hd command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.










    share|improve this question



























      10












      10








      10


      1






      I was solving a challenge where I found a data file with no file extension. The file command shows that it is a data file (application/octet-stream). The hd command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.










      share|improve this question
















      I was solving a challenge where I found a data file with no file extension. The file command shows that it is a data file (application/octet-stream). The hd command shows GNP. in the last line. So if I reverse this file then I will get the .PNG format file, I searched everywhere but I didn't find a solution explaining how to reverse the content of a binary file.







      binary






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited 2 hours ago









      Solomon Ucko

      10916




      10916










      asked Jan 11 '18 at 17:44









      Prvt_YadvPrvt_Yadv

      3,35631531




      3,35631531






















          5 Answers
          5






          active

          oldest

          votes


















          7














          With xxd (from vim) and tac (from GNU coreutils, also tail -r on some systems):



          < file.gnp xxd -p -c1 | tac | xxd -p -r > file.png





          share|improve this answer


























          • Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

            – Iulian Onofrei
            May 12 '18 at 17:26











          • This is not a solution because it will mirror all the file.

            – Philippe Delteil
            Nov 21 '18 at 3:33



















          7














          With perl:



          <file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png




          • -a: awk mode were records are split into fields


          • -0777 -p: slurp-mode, file read into one $_ record, modified record printed afterwards.


          • -F: empty field separator, so fields are individual bytes


          • $_=reverse@F: output record is the concatenation of the list of fields (@F) reversed.






          share|improve this answer































            4














            In zsh (the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):



            zmodload zsh/mapfile
            (LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)




            • LC_ALL=C: characters are bytes


            • $mapfile[file.gnp]: content of file.gnp file


            • s::: split the string into its byte constituents


            • Oa: reverse Order on array subscript that array






            share|improve this answer


























            • zsh is not the only shell that can handle binary data.

              – fpmurphy
              Jan 12 '18 at 14:21



















            2














            Here is one way of reversing a binary file using ksh93. I have left the code "loose" to make it easier to understand.



            #!/bin/ksh93

            typeset -b byte

            redirect 3< image.gpj || exit 1

            eof=$(3<#((EOF)))

            read -r -u 3 -N 1 byte
            printf "%B" byte > image.jpg
            3<#((CUR - 1))

            while (( $(3<#) > 0 ))
            do
            read -r -u 3 -N 1 byte
            printf "%B" byte >> image.jpg
            3<#((CUR - 2))
            done

            read -r -u 3 -N 1 byte
            printf "%B" byte >> image.jpg

            redirect 3<&- || echo 'cannot close FD 3'

            exit 0





            share|improve this answer


























            • nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

              – Stéphane Chazelas
              Jan 12 '18 at 15:51













            • Note that the first read above should read nothing as it's done at the end of the file.

              – Stéphane Chazelas
              Jan 12 '18 at 15:58











            • Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

              – Stéphane Chazelas
              Jan 12 '18 at 16:00













            • @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

              – fpmurphy
              Jan 13 '18 at 5:49













            • Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

              – Stéphane Chazelas
              Jan 13 '18 at 9:23





















            0














            I tried the following:



            tac -rs '.' input.gnp > output.png


            The idea is to force 'tac' using any character as separator.
            I tried that on a binary file and it seemed to work but any confirmation would be appreciated.



            Main advantage is that it does not load file into memory.






            share|improve this answer
























            • Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

              – Stéphane Chazelas
              Nov 15 '18 at 20:23











            • LC_ALL=C tac -rs $'.\|n' seems to work though.

              – Stéphane Chazelas
              Nov 15 '18 at 20:25












            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "106"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: false,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: null,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f416401%2fhow-to-reverse-the-content-of-binary-file%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            5 Answers
            5






            active

            oldest

            votes








            5 Answers
            5






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            7














            With xxd (from vim) and tac (from GNU coreutils, also tail -r on some systems):



            < file.gnp xxd -p -c1 | tac | xxd -p -r > file.png





            share|improve this answer


























            • Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

              – Iulian Onofrei
              May 12 '18 at 17:26











            • This is not a solution because it will mirror all the file.

              – Philippe Delteil
              Nov 21 '18 at 3:33
















            7














            With xxd (from vim) and tac (from GNU coreutils, also tail -r on some systems):



            < file.gnp xxd -p -c1 | tac | xxd -p -r > file.png





            share|improve this answer


























            • Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

              – Iulian Onofrei
              May 12 '18 at 17:26











            • This is not a solution because it will mirror all the file.

              – Philippe Delteil
              Nov 21 '18 at 3:33














            7












            7








            7







            With xxd (from vim) and tac (from GNU coreutils, also tail -r on some systems):



            < file.gnp xxd -p -c1 | tac | xxd -p -r > file.png





            share|improve this answer















            With xxd (from vim) and tac (from GNU coreutils, also tail -r on some systems):



            < file.gnp xxd -p -c1 | tac | xxd -p -r > file.png






            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Jan 12 '18 at 6:53

























            answered Jan 11 '18 at 21:41









            Stéphane ChazelasStéphane Chazelas

            315k57599957




            315k57599957













            • Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

              – Iulian Onofrei
              May 12 '18 at 17:26











            • This is not a solution because it will mirror all the file.

              – Philippe Delteil
              Nov 21 '18 at 3:33



















            • Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

              – Iulian Onofrei
              May 12 '18 at 17:26











            • This is not a solution because it will mirror all the file.

              – Philippe Delteil
              Nov 21 '18 at 3:33

















            Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

            – Iulian Onofrei
            May 12 '18 at 17:26





            Is there any way for this to be combined with vi.stackexchange.com/a/2237/10649? I tried all kind of combinations with no luck :(

            – Iulian Onofrei
            May 12 '18 at 17:26













            This is not a solution because it will mirror all the file.

            – Philippe Delteil
            Nov 21 '18 at 3:33





            This is not a solution because it will mirror all the file.

            – Philippe Delteil
            Nov 21 '18 at 3:33













            7














            With perl:



            <file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png




            • -a: awk mode were records are split into fields


            • -0777 -p: slurp-mode, file read into one $_ record, modified record printed afterwards.


            • -F: empty field separator, so fields are individual bytes


            • $_=reverse@F: output record is the concatenation of the list of fields (@F) reversed.






            share|improve this answer




























              7














              With perl:



              <file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png




              • -a: awk mode were records are split into fields


              • -0777 -p: slurp-mode, file read into one $_ record, modified record printed afterwards.


              • -F: empty field separator, so fields are individual bytes


              • $_=reverse@F: output record is the concatenation of the list of fields (@F) reversed.






              share|improve this answer


























                7












                7








                7







                With perl:



                <file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png




                • -a: awk mode were records are split into fields


                • -0777 -p: slurp-mode, file read into one $_ record, modified record printed afterwards.


                • -F: empty field separator, so fields are individual bytes


                • $_=reverse@F: output record is the concatenation of the list of fields (@F) reversed.






                share|improve this answer













                With perl:



                <file.gnp perl -0777 -F -ape '$_=reverse@F' > file.png




                • -a: awk mode were records are split into fields


                • -0777 -p: slurp-mode, file read into one $_ record, modified record printed afterwards.


                • -F: empty field separator, so fields are individual bytes


                • $_=reverse@F: output record is the concatenation of the list of fields (@F) reversed.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Jan 11 '18 at 21:46









                Stéphane ChazelasStéphane Chazelas

                315k57599957




                315k57599957























                    4














                    In zsh (the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):



                    zmodload zsh/mapfile
                    (LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)




                    • LC_ALL=C: characters are bytes


                    • $mapfile[file.gnp]: content of file.gnp file


                    • s::: split the string into its byte constituents


                    • Oa: reverse Order on array subscript that array






                    share|improve this answer


























                    • zsh is not the only shell that can handle binary data.

                      – fpmurphy
                      Jan 12 '18 at 14:21
















                    4














                    In zsh (the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):



                    zmodload zsh/mapfile
                    (LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)




                    • LC_ALL=C: characters are bytes


                    • $mapfile[file.gnp]: content of file.gnp file


                    • s::: split the string into its byte constituents


                    • Oa: reverse Order on array subscript that array






                    share|improve this answer


























                    • zsh is not the only shell that can handle binary data.

                      – fpmurphy
                      Jan 12 '18 at 14:21














                    4












                    4








                    4







                    In zsh (the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):



                    zmodload zsh/mapfile
                    (LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)




                    • LC_ALL=C: characters are bytes


                    • $mapfile[file.gnp]: content of file.gnp file


                    • s::: split the string into its byte constituents


                    • Oa: reverse Order on array subscript that array






                    share|improve this answer















                    In zsh (the only shell that can internally deal with binary data (unless you want to consider ksh93's base64 encoding approach)):



                    zmodload zsh/mapfile
                    (LC_ALL=C; printf %s ${(s::Oa)mapfile[file.gnp]} > file.png)




                    • LC_ALL=C: characters are bytes


                    • $mapfile[file.gnp]: content of file.gnp file


                    • s::: split the string into its byte constituents


                    • Oa: reverse Order on array subscript that array







                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Jan 12 '18 at 14:29

























                    answered Jan 11 '18 at 21:36









                    Stéphane ChazelasStéphane Chazelas

                    315k57599957




                    315k57599957













                    • zsh is not the only shell that can handle binary data.

                      – fpmurphy
                      Jan 12 '18 at 14:21



















                    • zsh is not the only shell that can handle binary data.

                      – fpmurphy
                      Jan 12 '18 at 14:21

















                    zsh is not the only shell that can handle binary data.

                    – fpmurphy
                    Jan 12 '18 at 14:21





                    zsh is not the only shell that can handle binary data.

                    – fpmurphy
                    Jan 12 '18 at 14:21











                    2














                    Here is one way of reversing a binary file using ksh93. I have left the code "loose" to make it easier to understand.



                    #!/bin/ksh93

                    typeset -b byte

                    redirect 3< image.gpj || exit 1

                    eof=$(3<#((EOF)))

                    read -r -u 3 -N 1 byte
                    printf "%B" byte > image.jpg
                    3<#((CUR - 1))

                    while (( $(3<#) > 0 ))
                    do
                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg
                    3<#((CUR - 2))
                    done

                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg

                    redirect 3<&- || echo 'cannot close FD 3'

                    exit 0





                    share|improve this answer


























                    • nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:51













                    • Note that the first read above should read nothing as it's done at the end of the file.

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:58











                    • Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                      – Stéphane Chazelas
                      Jan 12 '18 at 16:00













                    • @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                      – fpmurphy
                      Jan 13 '18 at 5:49













                    • Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                      – Stéphane Chazelas
                      Jan 13 '18 at 9:23


















                    2














                    Here is one way of reversing a binary file using ksh93. I have left the code "loose" to make it easier to understand.



                    #!/bin/ksh93

                    typeset -b byte

                    redirect 3< image.gpj || exit 1

                    eof=$(3<#((EOF)))

                    read -r -u 3 -N 1 byte
                    printf "%B" byte > image.jpg
                    3<#((CUR - 1))

                    while (( $(3<#) > 0 ))
                    do
                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg
                    3<#((CUR - 2))
                    done

                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg

                    redirect 3<&- || echo 'cannot close FD 3'

                    exit 0





                    share|improve this answer


























                    • nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:51













                    • Note that the first read above should read nothing as it's done at the end of the file.

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:58











                    • Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                      – Stéphane Chazelas
                      Jan 12 '18 at 16:00













                    • @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                      – fpmurphy
                      Jan 13 '18 at 5:49













                    • Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                      – Stéphane Chazelas
                      Jan 13 '18 at 9:23
















                    2












                    2








                    2







                    Here is one way of reversing a binary file using ksh93. I have left the code "loose" to make it easier to understand.



                    #!/bin/ksh93

                    typeset -b byte

                    redirect 3< image.gpj || exit 1

                    eof=$(3<#((EOF)))

                    read -r -u 3 -N 1 byte
                    printf "%B" byte > image.jpg
                    3<#((CUR - 1))

                    while (( $(3<#) > 0 ))
                    do
                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg
                    3<#((CUR - 2))
                    done

                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg

                    redirect 3<&- || echo 'cannot close FD 3'

                    exit 0





                    share|improve this answer















                    Here is one way of reversing a binary file using ksh93. I have left the code "loose" to make it easier to understand.



                    #!/bin/ksh93

                    typeset -b byte

                    redirect 3< image.gpj || exit 1

                    eof=$(3<#((EOF)))

                    read -r -u 3 -N 1 byte
                    printf "%B" byte > image.jpg
                    3<#((CUR - 1))

                    while (( $(3<#) > 0 ))
                    do
                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg
                    3<#((CUR - 2))
                    done

                    read -r -u 3 -N 1 byte
                    printf "%B" byte >> image.jpg

                    redirect 3<&- || echo 'cannot close FD 3'

                    exit 0






                    share|improve this answer














                    share|improve this answer



                    share|improve this answer








                    edited Jan 12 '18 at 14:23

























                    answered Jan 12 '18 at 14:15









                    fpmurphyfpmurphy

                    2,476916




                    2,476916













                    • nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:51













                    • Note that the first read above should read nothing as it's done at the end of the file.

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:58











                    • Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                      – Stéphane Chazelas
                      Jan 12 '18 at 16:00













                    • @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                      – fpmurphy
                      Jan 13 '18 at 5:49













                    • Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                      – Stéphane Chazelas
                      Jan 13 '18 at 9:23





















                    • nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:51













                    • Note that the first read above should read nothing as it's done at the end of the file.

                      – Stéphane Chazelas
                      Jan 12 '18 at 15:58











                    • Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                      – Stéphane Chazelas
                      Jan 12 '18 at 16:00













                    • @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                      – fpmurphy
                      Jan 13 '18 at 5:49













                    • Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                      – Stéphane Chazelas
                      Jan 13 '18 at 9:23



















                    nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                    – Stéphane Chazelas
                    Jan 12 '18 at 15:51







                    nice. That's the only answer so far that doesn't involve storing the whole file in memory. However, it's terribly inefficient in that it makes several system calls for each byte of the file (and conversions to/from base64), so wouldn't be suitable for files that don't fit in memory either. On my machine, it processes files at about 10KB/s

                    – Stéphane Chazelas
                    Jan 12 '18 at 15:51















                    Note that the first read above should read nothing as it's done at the end of the file.

                    – Stéphane Chazelas
                    Jan 12 '18 at 15:58





                    Note that the first read above should read nothing as it's done at the end of the file.

                    – Stéphane Chazelas
                    Jan 12 '18 at 15:58













                    Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                    – Stéphane Chazelas
                    Jan 12 '18 at 16:00







                    Trying to understand why it was so slow, I tried running it under strace and ksh93 seems to be behaving very weirdly, where it seeks all over the place within the file and reads large amounts at the time. Maybe a variant of github.com/att/ast/issues/15

                    – Stéphane Chazelas
                    Jan 12 '18 at 16:00















                    @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                    – fpmurphy
                    Jan 13 '18 at 5:49







                    @StéphaneChazelas. No mystery as to why it is relatively slow. Within the loop it has to seek backwards each time it reads a byte. This can easily be significantly reduced by a factor of 20 or even more by reading and writing more than one byte at a time. The write side of things can similarly be optimized. Lots of other techniques are available to further speed things up. I will leave that exercise up to you.

                    – fpmurphy
                    Jan 13 '18 at 5:49















                    Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                    – Stéphane Chazelas
                    Jan 13 '18 at 9:23







                    Try strace on the script to see what I mean. ksh93 reads the files thousands of times over. For instance, before reading the first byte, it seeks 64KiB off the end of the file, reads 64KiB, then seeks before the last byte and reads 1 byte and does something similar for every byte. Note that what you can do with those base64 encoded strings is limited, so if you read more than one byte at a time, it's going to be more difficult to extract the individual bytes of that.

                    – Stéphane Chazelas
                    Jan 13 '18 at 9:23













                    0














                    I tried the following:



                    tac -rs '.' input.gnp > output.png


                    The idea is to force 'tac' using any character as separator.
                    I tried that on a binary file and it seemed to work but any confirmation would be appreciated.



                    Main advantage is that it does not load file into memory.






                    share|improve this answer
























                    • Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:23











                    • LC_ALL=C tac -rs $'.\|n' seems to work though.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:25
















                    0














                    I tried the following:



                    tac -rs '.' input.gnp > output.png


                    The idea is to force 'tac' using any character as separator.
                    I tried that on a binary file and it seemed to work but any confirmation would be appreciated.



                    Main advantage is that it does not load file into memory.






                    share|improve this answer
























                    • Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:23











                    • LC_ALL=C tac -rs $'.\|n' seems to work though.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:25














                    0












                    0








                    0







                    I tried the following:



                    tac -rs '.' input.gnp > output.png


                    The idea is to force 'tac' using any character as separator.
                    I tried that on a binary file and it seemed to work but any confirmation would be appreciated.



                    Main advantage is that it does not load file into memory.






                    share|improve this answer













                    I tried the following:



                    tac -rs '.' input.gnp > output.png


                    The idea is to force 'tac' using any character as separator.
                    I tried that on a binary file and it seemed to work but any confirmation would be appreciated.



                    Main advantage is that it does not load file into memory.







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Apr 16 '18 at 16:00









                    BouteilleBouteille

                    1




                    1













                    • Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:23











                    • LC_ALL=C tac -rs $'.\|n' seems to work though.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:25



















                    • Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:23











                    • LC_ALL=C tac -rs $'.\|n' seems to work though.

                      – Stéphane Chazelas
                      Nov 15 '18 at 20:25

















                    Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                    – Stéphane Chazelas
                    Nov 15 '18 at 20:23





                    Doesn't work for me (here with GNU tac 8.28) when the input contains newline characters. printf '1n2' | tac -rs . | od -vAn -tc outputs n 2 1 instead of 2 n 1. You'd also need LC_ALL=C or . could match multi-byte characters.

                    – Stéphane Chazelas
                    Nov 15 '18 at 20:23













                    LC_ALL=C tac -rs $'.\|n' seems to work though.

                    – Stéphane Chazelas
                    Nov 15 '18 at 20:25





                    LC_ALL=C tac -rs $'.\|n' seems to work though.

                    – Stéphane Chazelas
                    Nov 15 '18 at 20:25


















                    draft saved

                    draft discarded




















































                    Thanks for contributing an answer to Unix & Linux Stack Exchange!


                    • Please be sure to answer the question. Provide details and share your research!

                    But avoid



                    • Asking for help, clarification, or responding to other answers.

                    • Making statements based on opinion; back them up with references or personal experience.


                    To learn more, see our tips on writing great answers.




                    draft saved


                    draft discarded














                    StackExchange.ready(
                    function () {
                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f416401%2fhow-to-reverse-the-content-of-binary-file%23new-answer', 'question_page');
                    }
                    );

                    Post as a guest















                    Required, but never shown





















































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown

































                    Required, but never shown














                    Required, but never shown












                    Required, but never shown







                    Required, but never shown







                    Popular posts from this blog

                    Taj Mahal Inhaltsverzeichnis Aufbau | Geschichte | 350-Jahr-Feier | Heutige Bedeutung | Siehe auch |...

                    Baia Sprie Cuprins Etimologie | Istorie | Demografie | Politică și administrație | Arii naturale...

                    Nicolae Petrescu-Găină Cuprins Biografie | Opera | In memoriam | Varia | Controverse, incertitudini...