NVMe disk shows 80% io utilization, partitions show 0% io utilizationLow level disk i/o on linuxUnderstanding...

Why did Missandei say this?

Why is there a cap on 401k contributions?

My Sixteen Friendly Students

Crime rates in a post-scarcity economy

How do I minimise waste on a flight?

Gift for mentor after his thesis defense?

What are my options legally if NYC company is not paying salary?

99 coins into the sacks

Why is it wrong to *implement* myself a known, published, widely believed to be secure crypto algorithm?

When I add a cylinder, it doesn't even show up on my screen at all

Are wands in any sort of book going to be too much like Harry Potter?

Are there vaccine ingredients which may not be disclosed ("hidden", "trade secret", or similar)?

Why doesn't Dany protect her dragons better?

Would the rotation of the starfield from a ring station be too disorienting?

Sed operations are not working or might i am doing it wrong?

Can a character shove an enemy who is already prone?

Names of the Six Tastes

Sprout Reports plugin - How to output a Matrix field into a row

The unknown and unexplained in science fiction

When was it publicly revealed that a KH-11 spy satellite took pictures of the first Shuttle flight?

Is there an idiom that means "revealing a secret unintentionally"?

How is it believable that Euron could so easily pull off this ambush?

Why are thrust reversers not used down to taxi speeds?

Can the president of the United States be guilty of insider trading?



NVMe disk shows 80% io utilization, partitions show 0% io utilization


Low level disk i/o on linuxUnderstanding iostat with Linux software RAIDCurrent disk loadNVMe PCIe disk power cyclingMonitor temperature/IO of NVMe driveiostat disk utilization is full when there is no IO happeningFDISK doesn't show any partitions for a disk, trying to resize an LVM disk, running CentOSDisk io stat “averaged” over a period of timeGet Disk Stats based on different parametersIO wait time is higher than disk utilization. Isn't this impossible?






.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty{ margin-bottom:0;
}







0















I have a CentOS 7 server (kernel 3.10.0-957.12.1.el7.x86_64) with 2 NVMe disks with the following setup:



# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
nvme0n1 259:0 0 477G 0 disk
├─nvme0n1p1 259:2 0 511M 0 part /boot/efi
├─nvme0n1p2 259:4 0 19.5G 0 part
│ └─md2 9:2 0 19.5G 0 raid1 /
├─nvme0n1p3 259:7 0 511M 0 part [SWAP]
└─nvme0n1p4 259:9 0 456.4G 0 part
└─data-data 253:0 0 912.8G 0 lvm /data
nvme1n1 259:1 0 477G 0 disk
├─nvme1n1p1 259:3 0 511M 0 part
├─nvme1n1p2 259:5 0 19.5G 0 part
│ └─md2 9:2 0 19.5G 0 raid1 /
├─nvme1n1p3 259:6 0 511M 0 part [SWAP]
└─nvme1n1p4 259:8 0 456.4G 0 part
└─data-data 253:0 0 912.8G 0 lvm /data


Our monitoring and iostat continually shows nvme0n1 and nvme1n1 with 80%+ io utilization while the individual partitions have 0% io utilization and are fully available (250k iops, 1GB read/write per sec).



avg-cpu:  %user   %nice %system %iowait  %steal   %idle
7.14 0.00 3.51 0.00 0.00 89.36

Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
nvme1n1 0.00 0.00 0.00 50.50 0.00 222.00 8.79 0.73 0.02 0.00 0.02 14.48 73.10
nvme1n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme1n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
nvme1n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme1n1p4 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme0n1 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.73 0.02 0.00 0.02 14.77 73.10
nvme0n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme0n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
nvme0n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
nvme0n1p4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
md2 0.00 0.00 0.00 48.50 0.00 214.00 8.82 0.00 0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00


Any ideas what can be the root cause for such behavior?

All seems to be working fine except for monitoring triggering high io alerts.










share|improve this question





























    0















    I have a CentOS 7 server (kernel 3.10.0-957.12.1.el7.x86_64) with 2 NVMe disks with the following setup:



    # lsblk
    NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    nvme0n1 259:0 0 477G 0 disk
    ├─nvme0n1p1 259:2 0 511M 0 part /boot/efi
    ├─nvme0n1p2 259:4 0 19.5G 0 part
    │ └─md2 9:2 0 19.5G 0 raid1 /
    ├─nvme0n1p3 259:7 0 511M 0 part [SWAP]
    └─nvme0n1p4 259:9 0 456.4G 0 part
    └─data-data 253:0 0 912.8G 0 lvm /data
    nvme1n1 259:1 0 477G 0 disk
    ├─nvme1n1p1 259:3 0 511M 0 part
    ├─nvme1n1p2 259:5 0 19.5G 0 part
    │ └─md2 9:2 0 19.5G 0 raid1 /
    ├─nvme1n1p3 259:6 0 511M 0 part [SWAP]
    └─nvme1n1p4 259:8 0 456.4G 0 part
    └─data-data 253:0 0 912.8G 0 lvm /data


    Our monitoring and iostat continually shows nvme0n1 and nvme1n1 with 80%+ io utilization while the individual partitions have 0% io utilization and are fully available (250k iops, 1GB read/write per sec).



    avg-cpu:  %user   %nice %system %iowait  %steal   %idle
    7.14 0.00 3.51 0.00 0.00 89.36

    Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
    nvme1n1 0.00 0.00 0.00 50.50 0.00 222.00 8.79 0.73 0.02 0.00 0.02 14.48 73.10
    nvme1n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    nvme1n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
    nvme1n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    nvme1n1p4 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00
    nvme0n1 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.73 0.02 0.00 0.02 14.77 73.10
    nvme0n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    nvme0n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
    nvme0n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    nvme0n1p4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
    md2 0.00 0.00 0.00 48.50 0.00 214.00 8.82 0.00 0.00 0.00 0.00 0.00 0.00
    dm-0 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00


    Any ideas what can be the root cause for such behavior?

    All seems to be working fine except for monitoring triggering high io alerts.










    share|improve this question

























      0












      0








      0








      I have a CentOS 7 server (kernel 3.10.0-957.12.1.el7.x86_64) with 2 NVMe disks with the following setup:



      # lsblk
      NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
      nvme0n1 259:0 0 477G 0 disk
      ├─nvme0n1p1 259:2 0 511M 0 part /boot/efi
      ├─nvme0n1p2 259:4 0 19.5G 0 part
      │ └─md2 9:2 0 19.5G 0 raid1 /
      ├─nvme0n1p3 259:7 0 511M 0 part [SWAP]
      └─nvme0n1p4 259:9 0 456.4G 0 part
      └─data-data 253:0 0 912.8G 0 lvm /data
      nvme1n1 259:1 0 477G 0 disk
      ├─nvme1n1p1 259:3 0 511M 0 part
      ├─nvme1n1p2 259:5 0 19.5G 0 part
      │ └─md2 9:2 0 19.5G 0 raid1 /
      ├─nvme1n1p3 259:6 0 511M 0 part [SWAP]
      └─nvme1n1p4 259:8 0 456.4G 0 part
      └─data-data 253:0 0 912.8G 0 lvm /data


      Our monitoring and iostat continually shows nvme0n1 and nvme1n1 with 80%+ io utilization while the individual partitions have 0% io utilization and are fully available (250k iops, 1GB read/write per sec).



      avg-cpu:  %user   %nice %system %iowait  %steal   %idle
      7.14 0.00 3.51 0.00 0.00 89.36

      Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
      nvme1n1 0.00 0.00 0.00 50.50 0.00 222.00 8.79 0.73 0.02 0.00 0.02 14.48 73.10
      nvme1n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme1n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
      nvme1n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme1n1p4 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.73 0.02 0.00 0.02 14.77 73.10
      nvme0n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
      nvme0n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1p4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      md2 0.00 0.00 0.00 48.50 0.00 214.00 8.82 0.00 0.00 0.00 0.00 0.00 0.00
      dm-0 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00


      Any ideas what can be the root cause for such behavior?

      All seems to be working fine except for monitoring triggering high io alerts.










      share|improve this question














      I have a CentOS 7 server (kernel 3.10.0-957.12.1.el7.x86_64) with 2 NVMe disks with the following setup:



      # lsblk
      NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
      nvme0n1 259:0 0 477G 0 disk
      ├─nvme0n1p1 259:2 0 511M 0 part /boot/efi
      ├─nvme0n1p2 259:4 0 19.5G 0 part
      │ └─md2 9:2 0 19.5G 0 raid1 /
      ├─nvme0n1p3 259:7 0 511M 0 part [SWAP]
      └─nvme0n1p4 259:9 0 456.4G 0 part
      └─data-data 253:0 0 912.8G 0 lvm /data
      nvme1n1 259:1 0 477G 0 disk
      ├─nvme1n1p1 259:3 0 511M 0 part
      ├─nvme1n1p2 259:5 0 19.5G 0 part
      │ └─md2 9:2 0 19.5G 0 raid1 /
      ├─nvme1n1p3 259:6 0 511M 0 part [SWAP]
      └─nvme1n1p4 259:8 0 456.4G 0 part
      └─data-data 253:0 0 912.8G 0 lvm /data


      Our monitoring and iostat continually shows nvme0n1 and nvme1n1 with 80%+ io utilization while the individual partitions have 0% io utilization and are fully available (250k iops, 1GB read/write per sec).



      avg-cpu:  %user   %nice %system %iowait  %steal   %idle
      7.14 0.00 3.51 0.00 0.00 89.36

      Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
      nvme1n1 0.00 0.00 0.00 50.50 0.00 222.00 8.79 0.73 0.02 0.00 0.02 14.48 73.10
      nvme1n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme1n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
      nvme1n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme1n1p4 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.73 0.02 0.00 0.02 14.77 73.10
      nvme0n1p1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1p2 0.00 0.00 0.00 49.50 0.00 218.00 8.81 0.00 0.02 0.00 0.02 0.01 0.05
      nvme0n1p3 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      nvme0n1p4 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
      md2 0.00 0.00 0.00 48.50 0.00 214.00 8.82 0.00 0.00 0.00 0.00 0.00 0.00
      dm-0 0.00 0.00 0.00 1.00 0.00 4.00 8.00 0.00 0.00 0.00 0.00 0.00 0.00


      Any ideas what can be the root cause for such behavior?

      All seems to be working fine except for monitoring triggering high io alerts.







      lvm performance io ssd iostat






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked 3 hours ago









      michal kralikmichal kralik

      1284




      1284






















          0






          active

          oldest

          votes












          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "106"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f517667%2fnvme-disk-shows-80-io-utilization-partitions-show-0-io-utilization%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes
















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Unix & Linux Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f517667%2fnvme-disk-shows-80-io-utilization-partitions-show-0-io-utilization%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Taj Mahal Inhaltsverzeichnis Aufbau | Geschichte | 350-Jahr-Feier | Heutige Bedeutung | Siehe auch |...

          Baia Sprie Cuprins Etimologie | Istorie | Demografie | Politică și administrație | Arii naturale...

          Nicolae Petrescu-Găină Cuprins Biografie | Opera | In memoriam | Varia | Controverse, incertitudini...