Mdthbs

What is the purpose of this mount on the upper suspension arm?

Does this put me at risk for identity theft?

Count number of occurences of particular numbers in list

Why do proponents of guns oppose gun competency tests?

Why can I log in to my Facebook account with a misspelled email/password?

Our group keeps dying during the Lost Mine of Phandelver campaign. What are we doing wrong?

Can you use the Help action to give a 2019 UA Artillerist artificer's turret advantage?

Who is the god Ao?

Can ads on a page read my password?

Is it double speak?

Why is Chromosome 1 called Chromosome 1?

I was contacted by a private bank overseas to get my inheritance

What can make Linux so unresponsive?

What is the bio-mechanical plausibility of a fox with venomous fangs?

What does VB stand for?

Why does putting a dot after the URL remove login information?

Does the length of a password for Wi-Fi affect speed?

Do any languages mention the top limit of a range first?

How many years before enough atoms of your body are replaced to survive the sudden disappearance of the original body’s atoms?

Did silent film actors actually say their lines or did they simply improvise “dialogue” while being filmed?

Colleagues speaking another language and it impacts work

Did WWII Japanese soldiers engage in cannibalism of their enemies?

Print only the last three columns from file

Was Richard I's imprisonment by Leopold of Austria justified?

Why ReLU function is not differentiable at 0?

Why is this function differentiable?Determining values where a function is not differentiableIs it possible that $f$ is differentiable?Why can a discontinuous function not be differentiable?Why is this function not differentiable at $(0,0)$Why is $f(x)$ not differentiable at $0$?How to prove the given function is not differentiable analytically?Why isn't this function differentiable?Finding where a function is not differentiableIs this non monotonic function differentiable at $x=1$?

.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}

3

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{begin{matrix}
0 quad if ; x leq 0\
x quad if ; x > 0
end{matrix}right.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

add a comment | 

3

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{begin{matrix}
0 quad if ; x leq 0\
x quad if ; x > 0
end{matrix}right.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

add a comment | 

$begingroup$

I'm kind of rusty in calculus.

Why the ReLU function is not differentiable at $f(0)$?

$
f(x) =
left{begin{matrix}
0 quad if ; x leq 0\
x quad if ; x > 0
end{matrix}right.
$

calculus

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

$endgroup$

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

share|cite|improve this question

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

asked 8 hours ago

Rodrigo VimieiroRodrigo Vimieiro

2055 bronze badges

2 Answers
2

active

oldest

votes

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$frac{df}{dx} = frac{d}{dx} x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$frac{df}{dx} = frac{d}{dx} 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

5

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_{h to 0} dfrac{f(0+h) - f(0)}{h}$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_{h to 0^+} dfrac {h-0}{h}=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_{h to 0^-} dfrac {0-0}{h}=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

Your Answer

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3316444%2fwhy-relu-function-is-not-differentiable-at-0%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$frac{df}{dx} = frac{d}{dx} x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$frac{df}{dx} = frac{d}{dx} 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

5

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$frac{df}{dx} = frac{d}{dx} x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$frac{df}{dx} = frac{d}{dx} 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

add a comment | 

$begingroup$

If you look at $x > 0$, or the righthand derivative, $$frac{df}{dx} = frac{d}{dx} x = 1$$ for all $x$.

If you look at $x le 0$, or the lefthand derivative, $$frac{df}{dx} = frac{d}{dx} 0 = 0$$ for all $x$.

Since $x = 0$ is the "break" point, the lefthand and righthand derivatives are not the same, and thus, the derivative is not defined at $x = 0$.

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

$endgroup$

share|cite|improve this answer

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

answered 8 hours ago

automaticallyGeneratedautomaticallyGenerated

1,13711 silver badge1111 bronze badges

5

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_{h to 0} dfrac{f(0+h) - f(0)}{h}$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_{h to 0^+} dfrac {h-0}{h}=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_{h to 0^-} dfrac {0-0}{h}=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

5

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_{h to 0} dfrac{f(0+h) - f(0)}{h}$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_{h to 0^+} dfrac {h-0}{h}=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_{h to 0^-} dfrac {0-0}{h}=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

add a comment | 

$begingroup$

Because it has a cusp at $0$ (a sharp corner), so it doesn't have a well defined tangent line; think about it, a tangent line is one which touches the graph only at one point. You can imagine many lines going through $(0, 0)$, so there are many possible tangent lines.

More formally, we have to investigate the limit

$$lim_{h to 0} dfrac{f(0+h) - f(0)}{h}$$

This limit does not exist for the function, because if you let $h$ approach $0$ from the right, you get

$$lim_{h to 0^+} dfrac {h-0}{h}=1.$$

While if you let $h$ approach $0$ from the left, that limit

$$lim_{h to 0^-} dfrac {0-0}{h}=0.$$

Therefore the limit does not exist, so the function is not differentiable at $0$.

share|cite|improve this answer

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

$endgroup$

share|cite|improve this answer

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

edited 8 hours ago

J. W. Tanner

13.1k11 gold badge99 silver badges2929 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges

answered 8 hours ago

OviOvi

13.3k1010 gold badges4545 silver badges121121 bronze badges