Pipe Fail (141) when piping output into tee — why?Default exit code when process is terminated?Print/Tee to...
How to "add" units to results of pgfmathsetmacro?
Are there any other rule mechanics that could grant Thieves' Cant?
Is there any way to stop a user from creating executables and running them?
Weird resistor with dots around it
Is this n-speak?
Why is the result of ('b'+'a'+ + 'a' + 'a').toLowerCase() 'banana'?
Clarification on Integrability
How can I see if the data in a SQL Server table is page-compressed?
What is the farthest a camera can see?
How can I find an old paper when the usual methods fail?
Why is Python 2.7 still the default Python version in Ubuntu?
Who invented Monoid?
What can Amex do if I cancel their card after using the sign up bonus miles?
PhD advisor lost funding, need advice
Can lodestones be used to magnetize crude iron weapons?
Symbol: Put a smile symbol under a plus
Corroded Metal vs Magical Armor, should it melt it?
A torrent of foreign terms
Escape Velocity - Won't the orbital path just become larger with higher initial velocity?
What is the hottest thing in the universe?
Did Pope Urban II issue the papal bull "terra nullius" in 1095?
Do I have to cite common CS algorithms?
How much can I judge a company based on a phone screening?
Word for an event that will likely never happen again
Pipe Fail (141) when piping output into tee — why?
Default exit code when process is terminated?Print/Tee to console without passing output to pipeDid the shellshock bash fix break basic command line tools like echo and cat?Why does history not overwrite but append when histappend is set to off in Bash?Preserve colours when piping to teeStrange output when using tee in pipe commandHow Do I Unbuffer The Output Passed From an Interactive Command Into a Pipeline Ending With `tee`?Redirection and piping for greppingTee does not write to file when combined to pipeInject string in tee outputtee into different variables
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}
An example should clarify my question. This behavior makes sense to me:
$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi
The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee
'd cat
, while another is printed by tee
's pass through pipe. So far, so good...
But what happens to the first "hi" in this case:
$ echo hi | tee >(echo yo)
yo
The return code is 141, a pipe fail. What could be causing this?
I'm running Mac OSX El Capitain, bash in default terminal app
bash tee
|
show 5 more comments
An example should clarify my question. This behavior makes sense to me:
$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi
The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee
'd cat
, while another is printed by tee
's pass through pipe. So far, so good...
But what happens to the first "hi" in this case:
$ echo hi | tee >(echo yo)
yo
The return code is 141, a pipe fail. What could be causing this?
I'm running Mac OSX El Capitain, bash in default terminal app
bash tee
2
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
3
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46
|
show 5 more comments
An example should clarify my question. This behavior makes sense to me:
$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi
The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee
'd cat
, while another is printed by tee
's pass through pipe. So far, so good...
But what happens to the first "hi" in this case:
$ echo hi | tee >(echo yo)
yo
The return code is 141, a pipe fail. What could be causing this?
I'm running Mac OSX El Capitain, bash in default terminal app
bash tee
An example should clarify my question. This behavior makes sense to me:
$ echo hi | cat
hi
$ echo hi | tee >(cat)
hi
hi
The first case is obvious. In the 2nd case, we pipe "hi" into tee using command substitution, and one "hi" is printed by the tee
'd cat
, while another is printed by tee
's pass through pipe. So far, so good...
But what happens to the first "hi" in this case:
$ echo hi | tee >(echo yo)
yo
The return code is 141, a pipe fail. What could be causing this?
I'm running Mac OSX El Capitain, bash in default terminal app
bash tee
bash tee
edited Apr 4 '16 at 3:55
Jonah
asked Apr 4 '16 at 3:09
JonahJonah
4841 gold badge6 silver badges13 bronze badges
4841 gold badge6 silver badges13 bronze badges
2
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
3
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46
|
show 5 more comments
2
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
3
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46
2
2
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
3
3
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46
|
show 5 more comments
2 Answers
2
active
oldest
votes
I think I’ve figured out how to tweak your experience
to turn it into something other people will be able to reproduce:
$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello … and, after a brief delay,
world
world
$ echo "$?"
0
$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello
$ echo "$?"
141
As you hopefully understand,
>(command)
creates a pipe
to a process running command
.
The standard input of command
is connected to a pathname
that other commands on the command line (in this case, tee
)
can open and write to. When command
is cat
,
the process sits there and reads from stdin until it gets an EOF.
In this case, tee
has no problem
writing all the data it reads from its stdin to the pipe.
But, when command
is echo yo
,
the process writes yo
to the stdout and immediately exits.
This causes a problem for tee
;
when it writes to a pipe with no process at the other end,
it gets a SIGPIPE signal.
Apparently OS X’s version of tee
writes to the file(s) on the command line first, and then its stdout.
So, in your example (echo hi | tee >(echo yo)
),
tee
gets the pipe failure on its very first write.
Whereas, the version of tee
on Linux and Cygwin
writes to the stdout first,
so it manages to write hi
to the screen before it dies.
In my enhanced example, tee
dies when it writes hello
to the pipe,
so it doesn’t get a chance to read and write world
.
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
add a comment |
To visualize what's going on, compare the following two variations:
bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
echo hi | wait_and_tee >(echo yo); echo $?'
Notice what's happening in the first variation?
$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
hi
0
$ yo
The command in the process substitution sleep 1; echo yo
is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:
- The three commands
echo hi
,sleep 1; echo yo
andtee
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).
tee
reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by>(…)
. This results in one copy ofhi
printed to the terminal, and one copy in the buffer of the>(…)
pipe.- In parallel with the previous bullet point,
echo hi
exits, which closes the write end of the|
pipe.
tee
notices that it has reached he end of its input file. It has written all of its data out, so it exits.- From bash's perspective, both sides of the pipe have exited, so the command is over. Since
tee
returned 0, the status of the pipeline is 0. - One second later,
sleep 1
finishes, andecho yo
is executed.
In the second variation, I force echo yo
to terminate before tee
, by forcing it to terminate before tee
starts. This time, the sequence of events is:
- The three commands
echo hi
,echo yo
andsleep 1
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).- In parallel with the previous bullet point,
echo yo
printsyo
and exits. - One second later,
sleep 1
exits andtee
starts.
tee
readshi
from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from>(…)
passed as an argument. Since the process that had this pipe open for reading (echo yo
) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).
As G-Man explains, whether hi
is displayed in the second case depends on whether tee
tries to write to its standard output or its file argument first.
Without the sleep
calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).
In my Debian, I always getyo
,hi
printed. There's always a SIGPIPE and return code is141
in both case. I guess the problem came from the lack of/dev/fd
in OSX.
– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of/dev/fd
on OSX matters.
– Gilles
Apr 5 '16 at 9:35
Because the lack of/dev/fd
causebash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the commandecho hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.
– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?
– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide thedtruss
output (I requested him in the comment but got no response).
– cuonglm
Apr 5 '16 at 9:52
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f274120%2fpipe-fail-141-when-piping-output-into-tee-why%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think I’ve figured out how to tweak your experience
to turn it into something other people will be able to reproduce:
$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello … and, after a brief delay,
world
world
$ echo "$?"
0
$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello
$ echo "$?"
141
As you hopefully understand,
>(command)
creates a pipe
to a process running command
.
The standard input of command
is connected to a pathname
that other commands on the command line (in this case, tee
)
can open and write to. When command
is cat
,
the process sits there and reads from stdin until it gets an EOF.
In this case, tee
has no problem
writing all the data it reads from its stdin to the pipe.
But, when command
is echo yo
,
the process writes yo
to the stdout and immediately exits.
This causes a problem for tee
;
when it writes to a pipe with no process at the other end,
it gets a SIGPIPE signal.
Apparently OS X’s version of tee
writes to the file(s) on the command line first, and then its stdout.
So, in your example (echo hi | tee >(echo yo)
),
tee
gets the pipe failure on its very first write.
Whereas, the version of tee
on Linux and Cygwin
writes to the stdout first,
so it manages to write hi
to the screen before it dies.
In my enhanced example, tee
dies when it writes hello
to the pipe,
so it doesn’t get a chance to read and write world
.
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
add a comment |
I think I’ve figured out how to tweak your experience
to turn it into something other people will be able to reproduce:
$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello … and, after a brief delay,
world
world
$ echo "$?"
0
$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello
$ echo "$?"
141
As you hopefully understand,
>(command)
creates a pipe
to a process running command
.
The standard input of command
is connected to a pathname
that other commands on the command line (in this case, tee
)
can open and write to. When command
is cat
,
the process sits there and reads from stdin until it gets an EOF.
In this case, tee
has no problem
writing all the data it reads from its stdin to the pipe.
But, when command
is echo yo
,
the process writes yo
to the stdout and immediately exits.
This causes a problem for tee
;
when it writes to a pipe with no process at the other end,
it gets a SIGPIPE signal.
Apparently OS X’s version of tee
writes to the file(s) on the command line first, and then its stdout.
So, in your example (echo hi | tee >(echo yo)
),
tee
gets the pipe failure on its very first write.
Whereas, the version of tee
on Linux and Cygwin
writes to the stdout first,
so it manages to write hi
to the screen before it dies.
In my enhanced example, tee
dies when it writes hello
to the pipe,
so it doesn’t get a chance to read and write world
.
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
add a comment |
I think I’ve figured out how to tweak your experience
to turn it into something other people will be able to reproduce:
$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello … and, after a brief delay,
world
world
$ echo "$?"
0
$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello
$ echo "$?"
141
As you hopefully understand,
>(command)
creates a pipe
to a process running command
.
The standard input of command
is connected to a pathname
that other commands on the command line (in this case, tee
)
can open and write to. When command
is cat
,
the process sits there and reads from stdin until it gets an EOF.
In this case, tee
has no problem
writing all the data it reads from its stdin to the pipe.
But, when command
is echo yo
,
the process writes yo
to the stdout and immediately exits.
This causes a problem for tee
;
when it writes to a pipe with no process at the other end,
it gets a SIGPIPE signal.
Apparently OS X’s version of tee
writes to the file(s) on the command line first, and then its stdout.
So, in your example (echo hi | tee >(echo yo)
),
tee
gets the pipe failure on its very first write.
Whereas, the version of tee
on Linux and Cygwin
writes to the stdout first,
so it manages to write hi
to the screen before it dies.
In my enhanced example, tee
dies when it writes hello
to the pipe,
so it doesn’t get a chance to read and write world
.
I think I’ve figured out how to tweak your experience
to turn it into something other people will be able to reproduce:
$ (echo hello; sleep 1; echo world) | tee >(cat)
hello
hello … and, after a brief delay,
world
world
$ echo "$?"
0
$ (echo hello; sleep 1; echo world) | tee >(echo yo)
yo
hello
$ echo "$?"
141
As you hopefully understand,
>(command)
creates a pipe
to a process running command
.
The standard input of command
is connected to a pathname
that other commands on the command line (in this case, tee
)
can open and write to. When command
is cat
,
the process sits there and reads from stdin until it gets an EOF.
In this case, tee
has no problem
writing all the data it reads from its stdin to the pipe.
But, when command
is echo yo
,
the process writes yo
to the stdout and immediately exits.
This causes a problem for tee
;
when it writes to a pipe with no process at the other end,
it gets a SIGPIPE signal.
Apparently OS X’s version of tee
writes to the file(s) on the command line first, and then its stdout.
So, in your example (echo hi | tee >(echo yo)
),
tee
gets the pipe failure on its very first write.
Whereas, the version of tee
on Linux and Cygwin
writes to the stdout first,
so it manages to write hi
to the screen before it dies.
In my enhanced example, tee
dies when it writes hello
to the pipe,
so it doesn’t get a chance to read and write world
.
edited Apr 4 '16 at 17:54
answered Apr 4 '16 at 5:25
G-ManG-Man
15.2k9 gold badges44 silver badges83 bronze badges
15.2k9 gold badges44 silver badges83 bronze badges
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
add a comment |
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
1
1
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
I don't think the problem is what you said. Try the command on linux, you will get 141 return code too. Try strace, and you see bash write to the file first. I guess the problem is the lack of /dev/fd, so bash used named pipe for process substitution, some thing went wrong in this case
– cuonglm
Apr 4 '16 at 18:47
add a comment |
To visualize what's going on, compare the following two variations:
bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
echo hi | wait_and_tee >(echo yo); echo $?'
Notice what's happening in the first variation?
$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
hi
0
$ yo
The command in the process substitution sleep 1; echo yo
is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:
- The three commands
echo hi
,sleep 1; echo yo
andtee
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).
tee
reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by>(…)
. This results in one copy ofhi
printed to the terminal, and one copy in the buffer of the>(…)
pipe.- In parallel with the previous bullet point,
echo hi
exits, which closes the write end of the|
pipe.
tee
notices that it has reached he end of its input file. It has written all of its data out, so it exits.- From bash's perspective, both sides of the pipe have exited, so the command is over. Since
tee
returned 0, the status of the pipeline is 0. - One second later,
sleep 1
finishes, andecho yo
is executed.
In the second variation, I force echo yo
to terminate before tee
, by forcing it to terminate before tee
starts. This time, the sequence of events is:
- The three commands
echo hi
,echo yo
andsleep 1
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).- In parallel with the previous bullet point,
echo yo
printsyo
and exits. - One second later,
sleep 1
exits andtee
starts.
tee
readshi
from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from>(…)
passed as an argument. Since the process that had this pipe open for reading (echo yo
) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).
As G-Man explains, whether hi
is displayed in the second case depends on whether tee
tries to write to its standard output or its file argument first.
Without the sleep
calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).
In my Debian, I always getyo
,hi
printed. There's always a SIGPIPE and return code is141
in both case. I guess the problem came from the lack of/dev/fd
in OSX.
– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of/dev/fd
on OSX matters.
– Gilles
Apr 5 '16 at 9:35
Because the lack of/dev/fd
causebash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the commandecho hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.
– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?
– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide thedtruss
output (I requested him in the comment but got no response).
– cuonglm
Apr 5 '16 at 9:52
add a comment |
To visualize what's going on, compare the following two variations:
bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
echo hi | wait_and_tee >(echo yo); echo $?'
Notice what's happening in the first variation?
$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
hi
0
$ yo
The command in the process substitution sleep 1; echo yo
is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:
- The three commands
echo hi
,sleep 1; echo yo
andtee
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).
tee
reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by>(…)
. This results in one copy ofhi
printed to the terminal, and one copy in the buffer of the>(…)
pipe.- In parallel with the previous bullet point,
echo hi
exits, which closes the write end of the|
pipe.
tee
notices that it has reached he end of its input file. It has written all of its data out, so it exits.- From bash's perspective, both sides of the pipe have exited, so the command is over. Since
tee
returned 0, the status of the pipeline is 0. - One second later,
sleep 1
finishes, andecho yo
is executed.
In the second variation, I force echo yo
to terminate before tee
, by forcing it to terminate before tee
starts. This time, the sequence of events is:
- The three commands
echo hi
,echo yo
andsleep 1
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).- In parallel with the previous bullet point,
echo yo
printsyo
and exits. - One second later,
sleep 1
exits andtee
starts.
tee
readshi
from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from>(…)
passed as an argument. Since the process that had this pipe open for reading (echo yo
) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).
As G-Man explains, whether hi
is displayed in the second case depends on whether tee
tries to write to its standard output or its file argument first.
Without the sleep
calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).
In my Debian, I always getyo
,hi
printed. There's always a SIGPIPE and return code is141
in both case. I guess the problem came from the lack of/dev/fd
in OSX.
– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of/dev/fd
on OSX matters.
– Gilles
Apr 5 '16 at 9:35
Because the lack of/dev/fd
causebash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the commandecho hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.
– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?
– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide thedtruss
output (I requested him in the comment but got no response).
– cuonglm
Apr 5 '16 at 9:52
add a comment |
To visualize what's going on, compare the following two variations:
bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
echo hi | wait_and_tee >(echo yo); echo $?'
Notice what's happening in the first variation?
$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
hi
0
$ yo
The command in the process substitution sleep 1; echo yo
is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:
- The three commands
echo hi
,sleep 1; echo yo
andtee
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).
tee
reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by>(…)
. This results in one copy ofhi
printed to the terminal, and one copy in the buffer of the>(…)
pipe.- In parallel with the previous bullet point,
echo hi
exits, which closes the write end of the|
pipe.
tee
notices that it has reached he end of its input file. It has written all of its data out, so it exits.- From bash's perspective, both sides of the pipe have exited, so the command is over. Since
tee
returned 0, the status of the pipeline is 0. - One second later,
sleep 1
finishes, andecho yo
is executed.
In the second variation, I force echo yo
to terminate before tee
, by forcing it to terminate before tee
starts. This time, the sequence of events is:
- The three commands
echo hi
,echo yo
andsleep 1
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).- In parallel with the previous bullet point,
echo yo
printsyo
and exits. - One second later,
sleep 1
exits andtee
starts.
tee
readshi
from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from>(…)
passed as an argument. Since the process that had this pipe open for reading (echo yo
) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).
As G-Man explains, whether hi
is displayed in the second case depends on whether tee
tries to write to its standard output or its file argument first.
Without the sleep
calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).
To visualize what's going on, compare the following two variations:
bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
bash -c 'wait_and_tee () { sleep 1; tee "$@"; };
echo hi | wait_and_tee >(echo yo); echo $?'
Notice what's happening in the first variation?
$ bash -c 'echo hi | tee >(sleep 1; echo yo); echo $?'
hi
0
$ yo
The command in the process substitution sleep 1; echo yo
is executed in parallel with the commands outside, and bash doesn't wait for it to finish. So the sequence of events is:
- The three commands
echo hi
,sleep 1; echo yo
andtee
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).
tee
reads from the pipe and writes to both its standard output and its command line argument, which is another pipe created by>(…)
. This results in one copy ofhi
printed to the terminal, and one copy in the buffer of the>(…)
pipe.- In parallel with the previous bullet point,
echo hi
exits, which closes the write end of the|
pipe.
tee
notices that it has reached he end of its input file. It has written all of its data out, so it exits.- From bash's perspective, both sides of the pipe have exited, so the command is over. Since
tee
returned 0, the status of the pipeline is 0. - One second later,
sleep 1
finishes, andecho yo
is executed.
In the second variation, I force echo yo
to terminate before tee
, by forcing it to terminate before tee
starts. This time, the sequence of events is:
- The three commands
echo hi
,echo yo
andsleep 1
are started in parallel.
echo hi
writeshi
to its output (the|
pipe).- In parallel with the previous bullet point,
echo yo
printsyo
and exits. - One second later,
sleep 1
exits andtee
starts.
tee
readshi
from its input, and attempts to write it both to the terminal (its standard output) and the pipe resulting from>(…)
passed as an argument. Since the process that had this pipe open for reading (echo yo
) exited a second ago, the attempt to write to the pipe fails with SIGPIPE (signal 13, which the shell reports as 128+signal_number).
As G-Man explains, whether hi
is displayed in the second case depends on whether tee
tries to write to its standard output or its file argument first.
Without the sleep
calls, the timing could go either way (I get about half/half under Linux, a different kernel might make one timing a lot more likely than the other).
edited Apr 13 '17 at 12:36
Community♦
1
1
answered Apr 5 '16 at 0:36
GillesGilles
569k136 gold badges1174 silver badges1686 bronze badges
569k136 gold badges1174 silver badges1686 bronze badges
In my Debian, I always getyo
,hi
printed. There's always a SIGPIPE and return code is141
in both case. I guess the problem came from the lack of/dev/fd
in OSX.
– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of/dev/fd
on OSX matters.
– Gilles
Apr 5 '16 at 9:35
Because the lack of/dev/fd
causebash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the commandecho hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.
– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?
– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide thedtruss
output (I requested him in the comment but got no response).
– cuonglm
Apr 5 '16 at 9:52
add a comment |
In my Debian, I always getyo
,hi
printed. There's always a SIGPIPE and return code is141
in both case. I guess the problem came from the lack of/dev/fd
in OSX.
– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of/dev/fd
on OSX matters.
– Gilles
Apr 5 '16 at 9:35
Because the lack of/dev/fd
causebash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the commandecho hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.
– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?
– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide thedtruss
output (I requested him in the comment but got no response).
– cuonglm
Apr 5 '16 at 9:52
In my Debian, I always get
yo
, hi
printed. There's always a SIGPIPE and return code is 141
in both case. I guess the problem came from the lack of /dev/fd
in OSX.– cuonglm
Apr 5 '16 at 1:47
In my Debian, I always get
yo
, hi
printed. There's always a SIGPIPE and return code is 141
in both case. I guess the problem came from the lack of /dev/fd
in OSX.– cuonglm
Apr 5 '16 at 1:47
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of
/dev/fd
on OSX matters.– Gilles
Apr 5 '16 at 9:35
@cuonglm On both Debian jessie and Ubuntu 14.04, I get either behavior randomly. I don't understand why you think the lack of
/dev/fd
on OSX matters.– Gilles
Apr 5 '16 at 9:35
Because the lack of
/dev/fd
cause bash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the command echo hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.– cuonglm
Apr 5 '16 at 9:40
Because the lack of
/dev/fd
cause bash
to use named pipe instead. I remember a bug was raised about the issue before bash 4.3 released. And in my above comment, I mean the command echo hi | tee >(echo yo)
always produce the consistent output in my Debian Jessie.– cuonglm
Apr 5 '16 at 9:40
@cuonglm I know that without
/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?– Gilles
Apr 5 '16 at 9:50
@cuonglm I know that without
/dev/fd
bash uses a named pipe instead of an anonymous one. And I know that in some cases it makes an observable difference, because opening and closing a pipe are observable events. But how does this change the possible behaviors in this case?– Gilles
Apr 5 '16 at 9:50
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide the
dtruss
output (I requested him in the comment but got no response).– cuonglm
Apr 5 '16 at 9:52
I'm not sure about this, that why I don't provide my own answer. I also don't have OSX now for tracing. It will be clearer if the OP can provide the
dtruss
output (I requested him in the comment but got no response).– cuonglm
Apr 5 '16 at 9:52
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f274120%2fpipe-fail-141-when-piping-output-into-tee-why%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
What system are you running? I am getting the correct output (both "yo" and "hi") on my systems (Arch, Cygwin, CentOS).
– nehcsivart
Apr 4 '16 at 3:18
3
can't reproduce..
– heemayl
Apr 4 '16 at 3:19
@tchen Mac OSX El Capitain, bash, running in default terminal app.
– Jonah
Apr 4 '16 at 3:20
@Jonah: What's the return code?
– cuonglm
Apr 4 '16 at 3:44
@heemayl, i verified on another system that the output is "yo hi". any idea what could be going on in my system to cause this? Mac uses BSD tee if that matters...
– Jonah
Apr 4 '16 at 3:46