Why is explainability not one of the criteria for publication?Editor indicated that if minor changes are...

Many many thanks

Which meaning of "must" does the Slow spell use?

Half filled water bottle

Units in general relativity

Stolen MacBook should I worry about my data?

Is there an in-universe explanation given to the senior Imperial Navy Officers as to why Darth Vader serves Emperor Palpatine?

Why is explainability not one of the criteria for publication?

Shift lens vs move body?

How to say "I only speak one which is English." in French?

What is the sound/audio equivalent of "unsightly"?

Did ancient peoples ever hide their treasure behind puzzles?

Notice period 60 days but I need to join in 45 days

How to pass 2>/dev/null as a variable?

Defending Castle from Zombies

Will removing shelving screws from studs damage the studs?

Term used to describe a person who predicts future outcomes

Why does a sticker slowly peel off, but if it is pulled quickly it tears?

Can an object tethered to a spaceship be pulled out of event horizon?

助けてくれて有難う meaning and usage

Recommended Breathing Exercises to Play Woodwinds

Unlock your Lock

How could a self contained organic body propel itself in space

What is Soda Fountain Etiquette?

Is there a word or phrase that means "use other people's wifi or Internet service without consent"?



Why is explainability not one of the criteria for publication?


Editor indicated that if minor changes are made, paper could be published: what are the chances of publication?How to ask summer research advisor for publication?Why are meta-analyses the most quoted type of research paper despite being often flawed?Found better technique just before publishing old, but still good technique – can the old one still be published?Why isn't implementing scientific libraries in software considered equivalent to a publication?Why is novelty mandatory for a Ph.D. degree?Will the PhD degree be retracted if the thesis is found to be not novel?






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







3















A paper is eligible for publishing in reputable journals in general if it satisfies the criteria objectivity, reproducibility and (optionally) novelty.



But why not they are considering Explainability as a criterion? Although the model proposed in the paper satisfies the above mentioned three metrics but not explainability, then how can it be considered as a contribution to field?



PS: Low "explainability" means proving something works without explaining how it works. See also "Interpretability"










share|improve this question






















  • 1





    If it didn't satisfy explanability, how did it get accepted by peer reviewers?

    – Coder
    10 hours ago






  • 1





    What's explainability? Do you mean accessibility?

    – user2768
    9 hours ago






  • 6





    So experimental results should not be published until they are well understood?

    – fqq
    9 hours ago






  • 3





    There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

    – Flyto
    4 hours ago






  • 1





    You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

    – Marco13
    1 hour ago


















3















A paper is eligible for publishing in reputable journals in general if it satisfies the criteria objectivity, reproducibility and (optionally) novelty.



But why not they are considering Explainability as a criterion? Although the model proposed in the paper satisfies the above mentioned three metrics but not explainability, then how can it be considered as a contribution to field?



PS: Low "explainability" means proving something works without explaining how it works. See also "Interpretability"










share|improve this question






















  • 1





    If it didn't satisfy explanability, how did it get accepted by peer reviewers?

    – Coder
    10 hours ago






  • 1





    What's explainability? Do you mean accessibility?

    – user2768
    9 hours ago






  • 6





    So experimental results should not be published until they are well understood?

    – fqq
    9 hours ago






  • 3





    There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

    – Flyto
    4 hours ago






  • 1





    You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

    – Marco13
    1 hour ago














3












3








3








A paper is eligible for publishing in reputable journals in general if it satisfies the criteria objectivity, reproducibility and (optionally) novelty.



But why not they are considering Explainability as a criterion? Although the model proposed in the paper satisfies the above mentioned three metrics but not explainability, then how can it be considered as a contribution to field?



PS: Low "explainability" means proving something works without explaining how it works. See also "Interpretability"










share|improve this question
















A paper is eligible for publishing in reputable journals in general if it satisfies the criteria objectivity, reproducibility and (optionally) novelty.



But why not they are considering Explainability as a criterion? Although the model proposed in the paper satisfies the above mentioned three metrics but not explainability, then how can it be considered as a contribution to field?



PS: Low "explainability" means proving something works without explaining how it works. See also "Interpretability"







publishability






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited 43 mins ago









Anonymous Physicist

25.7k9 gold badges52 silver badges107 bronze badges




25.7k9 gold badges52 silver badges107 bronze badges










asked 10 hours ago









hanugmhanugm

1,4212 gold badges17 silver badges25 bronze badges




1,4212 gold badges17 silver badges25 bronze badges











  • 1





    If it didn't satisfy explanability, how did it get accepted by peer reviewers?

    – Coder
    10 hours ago






  • 1





    What's explainability? Do you mean accessibility?

    – user2768
    9 hours ago






  • 6





    So experimental results should not be published until they are well understood?

    – fqq
    9 hours ago






  • 3





    There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

    – Flyto
    4 hours ago






  • 1





    You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

    – Marco13
    1 hour ago














  • 1





    If it didn't satisfy explanability, how did it get accepted by peer reviewers?

    – Coder
    10 hours ago






  • 1





    What's explainability? Do you mean accessibility?

    – user2768
    9 hours ago






  • 6





    So experimental results should not be published until they are well understood?

    – fqq
    9 hours ago






  • 3





    There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

    – Flyto
    4 hours ago






  • 1





    You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

    – Marco13
    1 hour ago








1




1





If it didn't satisfy explanability, how did it get accepted by peer reviewers?

– Coder
10 hours ago





If it didn't satisfy explanability, how did it get accepted by peer reviewers?

– Coder
10 hours ago




1




1





What's explainability? Do you mean accessibility?

– user2768
9 hours ago





What's explainability? Do you mean accessibility?

– user2768
9 hours ago




6




6





So experimental results should not be published until they are well understood?

– fqq
9 hours ago





So experimental results should not be published until they are well understood?

– fqq
9 hours ago




3




3





There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

– Flyto
4 hours ago





There are huge realms of knowledge where we know what happens, but not why it happens... are you proposing that that should not be publishable?

– Flyto
4 hours ago




1




1





You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

– Marco13
1 hour ago





You specifically mentioned neural networks - is this what the core of the question is aiming at? If so, I agree that we have a severe problem in computer sciences with people publishing thousands of black boxes in different sizes and shapes, and nobody dares to even try and understand what they are doing. Well, at least de.wikipedia.org/wiki/Explainable_Artificial_Intelligence has gained some attention recently. People are probably noticing by now that all this grew out of hand...

– Marco13
1 hour ago










4 Answers
4






active

oldest

votes


















9















Coming especially from a biomedical sciences perspective,




I mean proving something works without explaining how it works.




(from a comment describing what is meant by 'explainability')



this would be an absolute disaster for science. Many results are not explainable according to that criteria; many treatments are known to be successful without being explained. If we waited until findings were understood before publishing, science would move a lot more slowly.



If you had a black-box image processing algorithm that, for example, beat the state of the art in tumor detection in processing MRI images, that result would be very interesting and publishable without being able to explain the black-box. In fact, it would likely be unethical to not publish such a finding.






share|improve this answer





















  • 2





    For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

    – Marco13
    1 hour ago











  • @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

    – Bryan Krause
    1 hour ago













  • Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

    – Anonymous Physicist
    41 mins ago











  • @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

    – Ryan_L
    33 mins ago



















4















Papers are evaluated on a variety of criteria, including accessibility and the contribution to the field of research.



Now papers that not only report findings, but analyze findings and provide root causes for effects observed in the paper are obviously more valuable and are more likely to be accepted.



But from a scientific point of view, requiring that papers have this property would not be a good idea. Quite often, the root cause of an observed phenomenon is not known. Not being able to publish papers without finding the root cause would mean that information stays "unknown" until the person making a discovery also finds out the reason for an observed phenomenon, which could mean that it is never found out. For instance, if Mendel with his discovery that traits are inherited until the DNA was found,
that would have been quite a loss.



In computer science, you need to distinguish between pure theoretical computer and the rest. While in the former, the proofs provide all the reason you need, in the applied fields, at least part of the argument is some utility of the finding. There are many subfields in which algorithms are published that work well in practice despite not giving theoretical guarantees that they always work. Finding out why certain algorithms work well in practice would require to define exactly what "practice" means, which changes over time. Machine learning is a good example: we know that many machine learning algorithms can get stuck in local optima, and we have some ideas on how to prevent that (in many interesting cases). And then there is some theory that tries to capture this. But ultimately, the reason for why many of the approaches work are that the models to be learned are easy enough and the algorithm is good enough, which is very difficult to impossible to formalize to a level that it would be acceptable in a scientific paper. And then requiring an in-depth explanation of why a new approach works would essentially mean that there will be almost no publications of practical relevance.






share|improve this answer



































    1















    I'm not sure what you mean exactly by explainability and it cannot be a scientific metric if it doesn't exist in a dictionary.



    So I conclude what you are thinking about is that the content of an article has to explain something: an not well understood process, a new method, a new theory.



    Different fields have different standards and metrics. I'm sure there are different for publishing a new physical theory vs. an optimization of a machine learning algorithm for image recognition. But this is normally covered by the novelty and significance metric by a journal.



    From a philosophy of science point of view you also should see or inspect what the modus operandi of researchers in your field is. For example, in particle physics or cosmology researchers try to falsify the scientific paradigm/theory, especially if there are too many flaws in a currently used theory. I know some of the basics of machine learning theory and that many of it is based on mathematical methods developed in quantum physics. This is a bullet-proof theory pretty much, no one has falsified it until this day and physicists still try. But in engineering and even in applied physics depending on the topic/resarch question rather a positivistic modus operandi is used by researchers, e.g. optimizing/enhancing/backing up a machine learning algorithm without substantial questioning or falsification underlying theories. And for minor incremental improvements an explanation in the sense of why rather then how may be not necessary in your field and therefore no general metric if the underlying theories are not really touched. As soon as you question a theory or common measurement process, at least in physics, you need to input a good explanation in your article, why and how you do this. What is the motivation, why it is more accurate to describe something.



    When you say in the comment "proving something works without how it works", I think this is what sometimes in industrial machine learning happens, input - black box - output. But if you can neither explain how or why your algorithm works (better), in the best case you can call it smart engineering but not science that can/should be published ;-)






    share|improve this answer

































      -1















      There is another aspect to this that applies in some fields, even surprisingly diverse ones. It is "explainable to whom, exactly?" I'll use math as an example but it also applies to things like literary criticism and CS, I think.



      When a professional paper is written, it is written in such a way that people similar to the author can understand it. It isn't, normally, written for novices or people in other fields. The author(s) suspect that most of their readers will be just like themselves with a similar background and way of thinking. So a math proof, can, in many (most?) cases, leave out many steps that would make the paper more understandable to a novice, but would just slow down most of the readers.



      I think that any field, even one not as "arcane" as mathematics, but which has a large professional vocabulary that is well understood by experienced practitioners will have a lot of papers like this.



      On the other hand, people that write for a general audience may need to do just the opposite. Fill in more detail than professionals require and resort to metaphor and analogy more than experts need, just to be understood at all.



      Of course, the worst of all worlds is either0 to provide so much detail that the work becomes pedantic, pleasing no one or simply making unsupported statements requiring leaps of faith to follow (or not).



      In any case, what may be easily understood by you, may not be by myself, and vice-versa.



      Moreover, since the reviewers of any paper are probably a lot like the authors, then if they can understand it they won't object, and if they can't, then they will require modifications. So, your "requirement" is probably built into the process implicitly as member Coder implies in a comment.






      share|improve this answer






























        Your Answer








        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "415"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        noCode: true, onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f135305%2fwhy-is-explainability-not-one-of-the-criteria-for-publication%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        4 Answers
        4






        active

        oldest

        votes








        4 Answers
        4






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        9















        Coming especially from a biomedical sciences perspective,




        I mean proving something works without explaining how it works.




        (from a comment describing what is meant by 'explainability')



        this would be an absolute disaster for science. Many results are not explainable according to that criteria; many treatments are known to be successful without being explained. If we waited until findings were understood before publishing, science would move a lot more slowly.



        If you had a black-box image processing algorithm that, for example, beat the state of the art in tumor detection in processing MRI images, that result would be very interesting and publishable without being able to explain the black-box. In fact, it would likely be unethical to not publish such a finding.






        share|improve this answer





















        • 2





          For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

          – Marco13
          1 hour ago











        • @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

          – Bryan Krause
          1 hour ago













        • Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

          – Anonymous Physicist
          41 mins ago











        • @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

          – Ryan_L
          33 mins ago
















        9















        Coming especially from a biomedical sciences perspective,




        I mean proving something works without explaining how it works.




        (from a comment describing what is meant by 'explainability')



        this would be an absolute disaster for science. Many results are not explainable according to that criteria; many treatments are known to be successful without being explained. If we waited until findings were understood before publishing, science would move a lot more slowly.



        If you had a black-box image processing algorithm that, for example, beat the state of the art in tumor detection in processing MRI images, that result would be very interesting and publishable without being able to explain the black-box. In fact, it would likely be unethical to not publish such a finding.






        share|improve this answer





















        • 2





          For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

          – Marco13
          1 hour ago











        • @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

          – Bryan Krause
          1 hour ago













        • Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

          – Anonymous Physicist
          41 mins ago











        • @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

          – Ryan_L
          33 mins ago














        9














        9










        9









        Coming especially from a biomedical sciences perspective,




        I mean proving something works without explaining how it works.




        (from a comment describing what is meant by 'explainability')



        this would be an absolute disaster for science. Many results are not explainable according to that criteria; many treatments are known to be successful without being explained. If we waited until findings were understood before publishing, science would move a lot more slowly.



        If you had a black-box image processing algorithm that, for example, beat the state of the art in tumor detection in processing MRI images, that result would be very interesting and publishable without being able to explain the black-box. In fact, it would likely be unethical to not publish such a finding.






        share|improve this answer













        Coming especially from a biomedical sciences perspective,




        I mean proving something works without explaining how it works.




        (from a comment describing what is meant by 'explainability')



        this would be an absolute disaster for science. Many results are not explainable according to that criteria; many treatments are known to be successful without being explained. If we waited until findings were understood before publishing, science would move a lot more slowly.



        If you had a black-box image processing algorithm that, for example, beat the state of the art in tumor detection in processing MRI images, that result would be very interesting and publishable without being able to explain the black-box. In fact, it would likely be unethical to not publish such a finding.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered 8 hours ago









        Bryan KrauseBryan Krause

        20.7k5 gold badges63 silver badges82 bronze badges




        20.7k5 gold badges63 silver badges82 bronze badges











        • 2





          For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

          – Marco13
          1 hour ago











        • @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

          – Bryan Krause
          1 hour ago













        • Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

          – Anonymous Physicist
          41 mins ago











        • @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

          – Ryan_L
          33 mins ago














        • 2





          For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

          – Marco13
          1 hour ago











        • @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

          – Bryan Krause
          1 hour ago













        • Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

          – Anonymous Physicist
          41 mins ago











        • @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

          – Ryan_L
          33 mins ago








        2




        2





        For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

        – Marco13
        1 hour ago





        For the field of medical science your statement may be true, but that particular example from your last paragraph strikes a chord for me. Currently, we're seeing a flood of papers in computer science that apply "deep learning"/"neural networks" to a variety of highly specific problems, and do not say more than "It works (but we don't know why)". Many people are publishing like mad and literally do not have the slightest idea of what they are doing - even though that may seem hard to believe for other fields. Really, cf. xkcd.com/1838 : If it looks right, it will be published.

        – Marco13
        1 hour ago













        @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

        – Bryan Krause
        1 hour ago







        @Marco13 Indeed, though of course people still need to be cautious of xkcd.com/882 - that shouldn't be a barrier to publishing, though, it should be a barrier to how studies are interpreted and how results are validated on independent data. I should make clear that I have in mind things like paracetamol which are in wide use, definitely effective, and yet...still oddly not well understood.

        – Bryan Krause
        1 hour ago















        Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

        – Anonymous Physicist
        41 mins ago





        Superconductivity would be another example. New superconductors may require hundreds of papers before they are explained.

        – Anonymous Physicist
        41 mins ago













        @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

        – Ryan_L
        33 mins ago





        @Marco13 It's a horrible problem really. How can we know that the neural networks actually work if we don't know how they work? I know they had an AI that could check chest X-rays for tuberculosis, and while they did better than human x-ray technicians, many of the false positives it found were correctly assessed as negative by humans. The issue was the AI knew the xrays were coming from a hospital, it had learned this meant he or she was sicker, and this biased the test to think it was more likely tuberculosis. If you can't explain what patterns it's finding, how do you know they are real?

        – Ryan_L
        33 mins ago













        4















        Papers are evaluated on a variety of criteria, including accessibility and the contribution to the field of research.



        Now papers that not only report findings, but analyze findings and provide root causes for effects observed in the paper are obviously more valuable and are more likely to be accepted.



        But from a scientific point of view, requiring that papers have this property would not be a good idea. Quite often, the root cause of an observed phenomenon is not known. Not being able to publish papers without finding the root cause would mean that information stays "unknown" until the person making a discovery also finds out the reason for an observed phenomenon, which could mean that it is never found out. For instance, if Mendel with his discovery that traits are inherited until the DNA was found,
        that would have been quite a loss.



        In computer science, you need to distinguish between pure theoretical computer and the rest. While in the former, the proofs provide all the reason you need, in the applied fields, at least part of the argument is some utility of the finding. There are many subfields in which algorithms are published that work well in practice despite not giving theoretical guarantees that they always work. Finding out why certain algorithms work well in practice would require to define exactly what "practice" means, which changes over time. Machine learning is a good example: we know that many machine learning algorithms can get stuck in local optima, and we have some ideas on how to prevent that (in many interesting cases). And then there is some theory that tries to capture this. But ultimately, the reason for why many of the approaches work are that the models to be learned are easy enough and the algorithm is good enough, which is very difficult to impossible to formalize to a level that it would be acceptable in a scientific paper. And then requiring an in-depth explanation of why a new approach works would essentially mean that there will be almost no publications of practical relevance.






        share|improve this answer
































          4















          Papers are evaluated on a variety of criteria, including accessibility and the contribution to the field of research.



          Now papers that not only report findings, but analyze findings and provide root causes for effects observed in the paper are obviously more valuable and are more likely to be accepted.



          But from a scientific point of view, requiring that papers have this property would not be a good idea. Quite often, the root cause of an observed phenomenon is not known. Not being able to publish papers without finding the root cause would mean that information stays "unknown" until the person making a discovery also finds out the reason for an observed phenomenon, which could mean that it is never found out. For instance, if Mendel with his discovery that traits are inherited until the DNA was found,
          that would have been quite a loss.



          In computer science, you need to distinguish between pure theoretical computer and the rest. While in the former, the proofs provide all the reason you need, in the applied fields, at least part of the argument is some utility of the finding. There are many subfields in which algorithms are published that work well in practice despite not giving theoretical guarantees that they always work. Finding out why certain algorithms work well in practice would require to define exactly what "practice" means, which changes over time. Machine learning is a good example: we know that many machine learning algorithms can get stuck in local optima, and we have some ideas on how to prevent that (in many interesting cases). And then there is some theory that tries to capture this. But ultimately, the reason for why many of the approaches work are that the models to be learned are easy enough and the algorithm is good enough, which is very difficult to impossible to formalize to a level that it would be acceptable in a scientific paper. And then requiring an in-depth explanation of why a new approach works would essentially mean that there will be almost no publications of practical relevance.






          share|improve this answer






























            4














            4










            4









            Papers are evaluated on a variety of criteria, including accessibility and the contribution to the field of research.



            Now papers that not only report findings, but analyze findings and provide root causes for effects observed in the paper are obviously more valuable and are more likely to be accepted.



            But from a scientific point of view, requiring that papers have this property would not be a good idea. Quite often, the root cause of an observed phenomenon is not known. Not being able to publish papers without finding the root cause would mean that information stays "unknown" until the person making a discovery also finds out the reason for an observed phenomenon, which could mean that it is never found out. For instance, if Mendel with his discovery that traits are inherited until the DNA was found,
            that would have been quite a loss.



            In computer science, you need to distinguish between pure theoretical computer and the rest. While in the former, the proofs provide all the reason you need, in the applied fields, at least part of the argument is some utility of the finding. There are many subfields in which algorithms are published that work well in practice despite not giving theoretical guarantees that they always work. Finding out why certain algorithms work well in practice would require to define exactly what "practice" means, which changes over time. Machine learning is a good example: we know that many machine learning algorithms can get stuck in local optima, and we have some ideas on how to prevent that (in many interesting cases). And then there is some theory that tries to capture this. But ultimately, the reason for why many of the approaches work are that the models to be learned are easy enough and the algorithm is good enough, which is very difficult to impossible to formalize to a level that it would be acceptable in a scientific paper. And then requiring an in-depth explanation of why a new approach works would essentially mean that there will be almost no publications of practical relevance.






            share|improve this answer















            Papers are evaluated on a variety of criteria, including accessibility and the contribution to the field of research.



            Now papers that not only report findings, but analyze findings and provide root causes for effects observed in the paper are obviously more valuable and are more likely to be accepted.



            But from a scientific point of view, requiring that papers have this property would not be a good idea. Quite often, the root cause of an observed phenomenon is not known. Not being able to publish papers without finding the root cause would mean that information stays "unknown" until the person making a discovery also finds out the reason for an observed phenomenon, which could mean that it is never found out. For instance, if Mendel with his discovery that traits are inherited until the DNA was found,
            that would have been quite a loss.



            In computer science, you need to distinguish between pure theoretical computer and the rest. While in the former, the proofs provide all the reason you need, in the applied fields, at least part of the argument is some utility of the finding. There are many subfields in which algorithms are published that work well in practice despite not giving theoretical guarantees that they always work. Finding out why certain algorithms work well in practice would require to define exactly what "practice" means, which changes over time. Machine learning is a good example: we know that many machine learning algorithms can get stuck in local optima, and we have some ideas on how to prevent that (in many interesting cases). And then there is some theory that tries to capture this. But ultimately, the reason for why many of the approaches work are that the models to be learned are easy enough and the algorithm is good enough, which is very difficult to impossible to formalize to a level that it would be acceptable in a scientific paper. And then requiring an in-depth explanation of why a new approach works would essentially mean that there will be almost no publications of practical relevance.







            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited 9 hours ago

























            answered 9 hours ago









            DCTLibDCTLib

            9,36631 silver badges40 bronze badges




            9,36631 silver badges40 bronze badges


























                1















                I'm not sure what you mean exactly by explainability and it cannot be a scientific metric if it doesn't exist in a dictionary.



                So I conclude what you are thinking about is that the content of an article has to explain something: an not well understood process, a new method, a new theory.



                Different fields have different standards and metrics. I'm sure there are different for publishing a new physical theory vs. an optimization of a machine learning algorithm for image recognition. But this is normally covered by the novelty and significance metric by a journal.



                From a philosophy of science point of view you also should see or inspect what the modus operandi of researchers in your field is. For example, in particle physics or cosmology researchers try to falsify the scientific paradigm/theory, especially if there are too many flaws in a currently used theory. I know some of the basics of machine learning theory and that many of it is based on mathematical methods developed in quantum physics. This is a bullet-proof theory pretty much, no one has falsified it until this day and physicists still try. But in engineering and even in applied physics depending on the topic/resarch question rather a positivistic modus operandi is used by researchers, e.g. optimizing/enhancing/backing up a machine learning algorithm without substantial questioning or falsification underlying theories. And for minor incremental improvements an explanation in the sense of why rather then how may be not necessary in your field and therefore no general metric if the underlying theories are not really touched. As soon as you question a theory or common measurement process, at least in physics, you need to input a good explanation in your article, why and how you do this. What is the motivation, why it is more accurate to describe something.



                When you say in the comment "proving something works without how it works", I think this is what sometimes in industrial machine learning happens, input - black box - output. But if you can neither explain how or why your algorithm works (better), in the best case you can call it smart engineering but not science that can/should be published ;-)






                share|improve this answer






























                  1















                  I'm not sure what you mean exactly by explainability and it cannot be a scientific metric if it doesn't exist in a dictionary.



                  So I conclude what you are thinking about is that the content of an article has to explain something: an not well understood process, a new method, a new theory.



                  Different fields have different standards and metrics. I'm sure there are different for publishing a new physical theory vs. an optimization of a machine learning algorithm for image recognition. But this is normally covered by the novelty and significance metric by a journal.



                  From a philosophy of science point of view you also should see or inspect what the modus operandi of researchers in your field is. For example, in particle physics or cosmology researchers try to falsify the scientific paradigm/theory, especially if there are too many flaws in a currently used theory. I know some of the basics of machine learning theory and that many of it is based on mathematical methods developed in quantum physics. This is a bullet-proof theory pretty much, no one has falsified it until this day and physicists still try. But in engineering and even in applied physics depending on the topic/resarch question rather a positivistic modus operandi is used by researchers, e.g. optimizing/enhancing/backing up a machine learning algorithm without substantial questioning or falsification underlying theories. And for minor incremental improvements an explanation in the sense of why rather then how may be not necessary in your field and therefore no general metric if the underlying theories are not really touched. As soon as you question a theory or common measurement process, at least in physics, you need to input a good explanation in your article, why and how you do this. What is the motivation, why it is more accurate to describe something.



                  When you say in the comment "proving something works without how it works", I think this is what sometimes in industrial machine learning happens, input - black box - output. But if you can neither explain how or why your algorithm works (better), in the best case you can call it smart engineering but not science that can/should be published ;-)






                  share|improve this answer




























                    1














                    1










                    1









                    I'm not sure what you mean exactly by explainability and it cannot be a scientific metric if it doesn't exist in a dictionary.



                    So I conclude what you are thinking about is that the content of an article has to explain something: an not well understood process, a new method, a new theory.



                    Different fields have different standards and metrics. I'm sure there are different for publishing a new physical theory vs. an optimization of a machine learning algorithm for image recognition. But this is normally covered by the novelty and significance metric by a journal.



                    From a philosophy of science point of view you also should see or inspect what the modus operandi of researchers in your field is. For example, in particle physics or cosmology researchers try to falsify the scientific paradigm/theory, especially if there are too many flaws in a currently used theory. I know some of the basics of machine learning theory and that many of it is based on mathematical methods developed in quantum physics. This is a bullet-proof theory pretty much, no one has falsified it until this day and physicists still try. But in engineering and even in applied physics depending on the topic/resarch question rather a positivistic modus operandi is used by researchers, e.g. optimizing/enhancing/backing up a machine learning algorithm without substantial questioning or falsification underlying theories. And for minor incremental improvements an explanation in the sense of why rather then how may be not necessary in your field and therefore no general metric if the underlying theories are not really touched. As soon as you question a theory or common measurement process, at least in physics, you need to input a good explanation in your article, why and how you do this. What is the motivation, why it is more accurate to describe something.



                    When you say in the comment "proving something works without how it works", I think this is what sometimes in industrial machine learning happens, input - black box - output. But if you can neither explain how or why your algorithm works (better), in the best case you can call it smart engineering but not science that can/should be published ;-)






                    share|improve this answer













                    I'm not sure what you mean exactly by explainability and it cannot be a scientific metric if it doesn't exist in a dictionary.



                    So I conclude what you are thinking about is that the content of an article has to explain something: an not well understood process, a new method, a new theory.



                    Different fields have different standards and metrics. I'm sure there are different for publishing a new physical theory vs. an optimization of a machine learning algorithm for image recognition. But this is normally covered by the novelty and significance metric by a journal.



                    From a philosophy of science point of view you also should see or inspect what the modus operandi of researchers in your field is. For example, in particle physics or cosmology researchers try to falsify the scientific paradigm/theory, especially if there are too many flaws in a currently used theory. I know some of the basics of machine learning theory and that many of it is based on mathematical methods developed in quantum physics. This is a bullet-proof theory pretty much, no one has falsified it until this day and physicists still try. But in engineering and even in applied physics depending on the topic/resarch question rather a positivistic modus operandi is used by researchers, e.g. optimizing/enhancing/backing up a machine learning algorithm without substantial questioning or falsification underlying theories. And for minor incremental improvements an explanation in the sense of why rather then how may be not necessary in your field and therefore no general metric if the underlying theories are not really touched. As soon as you question a theory or common measurement process, at least in physics, you need to input a good explanation in your article, why and how you do this. What is the motivation, why it is more accurate to describe something.



                    When you say in the comment "proving something works without how it works", I think this is what sometimes in industrial machine learning happens, input - black box - output. But if you can neither explain how or why your algorithm works (better), in the best case you can call it smart engineering but not science that can/should be published ;-)







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered 9 hours ago









                    serasera

                    2,5375 silver badges16 bronze badges




                    2,5375 silver badges16 bronze badges


























                        -1















                        There is another aspect to this that applies in some fields, even surprisingly diverse ones. It is "explainable to whom, exactly?" I'll use math as an example but it also applies to things like literary criticism and CS, I think.



                        When a professional paper is written, it is written in such a way that people similar to the author can understand it. It isn't, normally, written for novices or people in other fields. The author(s) suspect that most of their readers will be just like themselves with a similar background and way of thinking. So a math proof, can, in many (most?) cases, leave out many steps that would make the paper more understandable to a novice, but would just slow down most of the readers.



                        I think that any field, even one not as "arcane" as mathematics, but which has a large professional vocabulary that is well understood by experienced practitioners will have a lot of papers like this.



                        On the other hand, people that write for a general audience may need to do just the opposite. Fill in more detail than professionals require and resort to metaphor and analogy more than experts need, just to be understood at all.



                        Of course, the worst of all worlds is either0 to provide so much detail that the work becomes pedantic, pleasing no one or simply making unsupported statements requiring leaps of faith to follow (or not).



                        In any case, what may be easily understood by you, may not be by myself, and vice-versa.



                        Moreover, since the reviewers of any paper are probably a lot like the authors, then if they can understand it they won't object, and if they can't, then they will require modifications. So, your "requirement" is probably built into the process implicitly as member Coder implies in a comment.






                        share|improve this answer
































                          -1















                          There is another aspect to this that applies in some fields, even surprisingly diverse ones. It is "explainable to whom, exactly?" I'll use math as an example but it also applies to things like literary criticism and CS, I think.



                          When a professional paper is written, it is written in such a way that people similar to the author can understand it. It isn't, normally, written for novices or people in other fields. The author(s) suspect that most of their readers will be just like themselves with a similar background and way of thinking. So a math proof, can, in many (most?) cases, leave out many steps that would make the paper more understandable to a novice, but would just slow down most of the readers.



                          I think that any field, even one not as "arcane" as mathematics, but which has a large professional vocabulary that is well understood by experienced practitioners will have a lot of papers like this.



                          On the other hand, people that write for a general audience may need to do just the opposite. Fill in more detail than professionals require and resort to metaphor and analogy more than experts need, just to be understood at all.



                          Of course, the worst of all worlds is either0 to provide so much detail that the work becomes pedantic, pleasing no one or simply making unsupported statements requiring leaps of faith to follow (or not).



                          In any case, what may be easily understood by you, may not be by myself, and vice-versa.



                          Moreover, since the reviewers of any paper are probably a lot like the authors, then if they can understand it they won't object, and if they can't, then they will require modifications. So, your "requirement" is probably built into the process implicitly as member Coder implies in a comment.






                          share|improve this answer






























                            -1














                            -1










                            -1









                            There is another aspect to this that applies in some fields, even surprisingly diverse ones. It is "explainable to whom, exactly?" I'll use math as an example but it also applies to things like literary criticism and CS, I think.



                            When a professional paper is written, it is written in such a way that people similar to the author can understand it. It isn't, normally, written for novices or people in other fields. The author(s) suspect that most of their readers will be just like themselves with a similar background and way of thinking. So a math proof, can, in many (most?) cases, leave out many steps that would make the paper more understandable to a novice, but would just slow down most of the readers.



                            I think that any field, even one not as "arcane" as mathematics, but which has a large professional vocabulary that is well understood by experienced practitioners will have a lot of papers like this.



                            On the other hand, people that write for a general audience may need to do just the opposite. Fill in more detail than professionals require and resort to metaphor and analogy more than experts need, just to be understood at all.



                            Of course, the worst of all worlds is either0 to provide so much detail that the work becomes pedantic, pleasing no one or simply making unsupported statements requiring leaps of faith to follow (or not).



                            In any case, what may be easily understood by you, may not be by myself, and vice-versa.



                            Moreover, since the reviewers of any paper are probably a lot like the authors, then if they can understand it they won't object, and if they can't, then they will require modifications. So, your "requirement" is probably built into the process implicitly as member Coder implies in a comment.






                            share|improve this answer















                            There is another aspect to this that applies in some fields, even surprisingly diverse ones. It is "explainable to whom, exactly?" I'll use math as an example but it also applies to things like literary criticism and CS, I think.



                            When a professional paper is written, it is written in such a way that people similar to the author can understand it. It isn't, normally, written for novices or people in other fields. The author(s) suspect that most of their readers will be just like themselves with a similar background and way of thinking. So a math proof, can, in many (most?) cases, leave out many steps that would make the paper more understandable to a novice, but would just slow down most of the readers.



                            I think that any field, even one not as "arcane" as mathematics, but which has a large professional vocabulary that is well understood by experienced practitioners will have a lot of papers like this.



                            On the other hand, people that write for a general audience may need to do just the opposite. Fill in more detail than professionals require and resort to metaphor and analogy more than experts need, just to be understood at all.



                            Of course, the worst of all worlds is either0 to provide so much detail that the work becomes pedantic, pleasing no one or simply making unsupported statements requiring leaps of faith to follow (or not).



                            In any case, what may be easily understood by you, may not be by myself, and vice-versa.



                            Moreover, since the reviewers of any paper are probably a lot like the authors, then if they can understand it they won't object, and if they can't, then they will require modifications. So, your "requirement" is probably built into the process implicitly as member Coder implies in a comment.







                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited 8 hours ago

























                            answered 9 hours ago









                            BuffyBuffy

                            79.4k21 gold badges244 silver badges352 bronze badges




                            79.4k21 gold badges244 silver badges352 bronze badges

































                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Academia Stack Exchange!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2facademia.stackexchange.com%2fquestions%2f135305%2fwhy-is-explainability-not-one-of-the-criteria-for-publication%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Taj Mahal Inhaltsverzeichnis Aufbau | Geschichte | 350-Jahr-Feier | Heutige Bedeutung | Siehe auch |...

                                Baia Sprie Cuprins Etimologie | Istorie | Demografie | Politică și administrație | Arii naturale...

                                Nicolae Petrescu-Găină Cuprins Biografie | Opera | In memoriam | Varia | Controverse, incertitudini...