Running on Redhat 5.1 if I do, dd bs=1024 count=1000000 if=/dev/zero of=/dev/null I get around 30Gb/sec However, when I do this with 2.6.37 I get close to 5GB/sec Is there anything new in the kernel? TIA
Hi... On Thu, Jan 20, 2011 at 10:36, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do,
Are you sure you're using that archaic distro? Or are you talking about RHEL 5.1?
dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
Hm, mine is: $ dd bs=1024 count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 1.12169 seconds, 913 MB/s This is on 2.6.36 SMP kernel compiled with gcc version 4.1.2 20080704 (Red Hat 4.1.2-48).
However, when I do this with 2.6.37 I get close to 5GB/sec
what if you use another blocksize, let's say 4K or even 32K? here's mine (again): $ dd bs=4K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 4096000000 bytes (4.1 GB) copied, 1.31167 seconds, 3.1 GB/s $ dd bs=32K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 32768000000 bytes (33 GB) copied, 4.91775 seconds, 6.7 GB/s see the difference? IMHO it's a matter of what I call "block merge efficiency"....the more you stuff pages (that fits into a "magic" number), the faster I/O you got. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
I am using RHEL 5.1. Sorry for not being clear. I wil give this a try today when I go back to class. But I was just curious why this was happening. How would one disable "block merge" efficiency? Has this feature been added recently? On Thu, Jan 20, 2011 at 12:28 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On Thu, Jan 20, 2011 at 10:36, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do,
Are you sure you're using that archaic distro? Or are you talking about RHEL 5.1?
dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
Hm, mine is: $ dd bs=1024 count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 1.12169 seconds, 913 MB/s
This is on 2.6.36 SMP kernel compiled with gcc version 4.1.2 20080704 (Red Hat 4.1.2-48).
However, when I do this with 2.6.37 I get close to 5GB/sec
what if you use another blocksize, let's say 4K or even 32K? here's mine (again): $ dd bs=4K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 4096000000 bytes (4.1 GB) copied, 1.31167 seconds, 3.1 GB/s
$ dd bs=32K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 32768000000 bytes (33 GB) copied, 4.91775 seconds, 6.7 GB/s
see the difference?
IMHO it's a matter of what I call "block merge efficiency"....the more you stuff pages (that fits into a "magic" number), the faster I/O you got.
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Mulyadi, You disappoint me. ;( Just kidding, but discussing dd throughput without the "conv=fdatasync" parameter is just a waste of everyone's time. And Mag, use a big enough count that it at least takes a few seconds to complete. A tenth of a second or less is just way to short to use as a benchmark. Greg On Thu, Jan 20, 2011 at 12:28 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On Thu, Jan 20, 2011 at 10:36, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do,
Are you sure you're using that archaic distro? Or are you talking about RHEL 5.1?
dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
Hm, mine is: $ dd bs=1024 count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 1.12169 seconds, 913 MB/s
This is on 2.6.36 SMP kernel compiled with gcc version 4.1.2 20080704 (Red Hat 4.1.2-48).
However, when I do this with 2.6.37 I get close to 5GB/sec
what if you use another blocksize, let's say 4K or even 32K? here's mine (again): $ dd bs=4K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 4096000000 bytes (4.1 GB) copied, 1.31167 seconds, 3.1 GB/s
$ dd bs=32K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 32768000000 bytes (33 GB) copied, 4.91775 seconds, 6.7 GB/s
see the difference?
IMHO it's a matter of what I call "block merge efficiency"....the more you stuff pages (that fits into a "magic" number), the faster I/O you got.
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev... The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
Greg, Yes, I did one very big like 100TB and I still see the regression. I even tried it with your extra dd option. I am wondering if the new kernel (2.6.36) introduced an options I need to set ? Can someone else try this? To reiterate the test scenario, Box 1, RHEL 5.1, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 2, RHEL 5.2, stock kernel,dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 3, RHEL 5.3, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 4, RHEL 5.4, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 5, RHEL 5.4, 2.6.35, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 5 takes much much longer. And all of these boxes are the same model and specs... On Thu, Jan 20, 2011 at 9:29 AM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
Mulyadi,
You disappoint me. ;(
Just kidding, but discussing dd throughput without the "conv=fdatasync" parameter is just a waste of everyone's time.
And Mag, use a big enough count that it at least takes a few seconds to complete. A tenth of a second or less is just way to short to use as a benchmark.
Greg
On Thu, Jan 20, 2011 at 12:28 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On Thu, Jan 20, 2011 at 10:36, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do,
Are you sure you're using that archaic distro? Or are you talking about RHEL 5.1?
dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
Hm, mine is: $ dd bs=1024 count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 1.12169 seconds, 913 MB/s
This is on 2.6.36 SMP kernel compiled with gcc version 4.1.2 20080704 (Red Hat 4.1.2-48).
However, when I do this with 2.6.37 I get close to 5GB/sec
what if you use another blocksize, let's say 4K or even 32K? here's mine (again): $ dd bs=4K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 4096000000 bytes (4.1 GB) copied, 1.31167 seconds, 3.1 GB/s
$ dd bs=32K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 32768000000 bytes (33 GB) copied, 4.91775 seconds, 6.7 GB/s
see the difference?
IMHO it's a matter of what I call "block merge efficiency"....the more you stuff pages (that fits into a "magic" number), the faster I/O you got.
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev...
The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
Typo, in Box 5. Wrong kernel version Box 5, RHEL 5.4, 2.6.36, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Anyone had a chance to try this? :-) On Thu, Jan 20, 2011 at 8:16 PM, Mag Gam <magawake@gmail.com> wrote:
Greg,
Yes, I did one very big like 100TB and I still see the regression. I even tried it with your extra dd option. I am wondering if the new kernel (2.6.36) introduced an options I need to set ?
Can someone else try this?
To reiterate the test scenario, Box 1, RHEL 5.1, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 2, RHEL 5.2, stock kernel,dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 3, RHEL 5.3, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 4, RHEL 5.4, stock kernel, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null Box 5, RHEL 5.4, 2.6.35, dd bs=1024 count=1000000 if=/dev/zero bs=4096k of=/dev/null
Box 5 takes much much longer.
And all of these boxes are the same model and specs...
On Thu, Jan 20, 2011 at 9:29 AM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
Mulyadi,
You disappoint me. ;(
Just kidding, but discussing dd throughput without the "conv=fdatasync" parameter is just a waste of everyone's time.
And Mag, use a big enough count that it at least takes a few seconds to complete. A tenth of a second or less is just way to short to use as a benchmark.
Greg
On Thu, Jan 20, 2011 at 12:28 AM, Mulyadi Santosa <mulyadi.santosa@gmail.com> wrote:
Hi...
On Thu, Jan 20, 2011 at 10:36, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do,
Are you sure you're using that archaic distro? Or are you talking about RHEL 5.1?
dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
Hm, mine is: $ dd bs=1024 count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 1024000000 bytes (1.0 GB) copied, 1.12169 seconds, 913 MB/s
This is on 2.6.36 SMP kernel compiled with gcc version 4.1.2 20080704 (Red Hat 4.1.2-48).
However, when I do this with 2.6.37 I get close to 5GB/sec
what if you use another blocksize, let's say 4K or even 32K? here's mine (again): $ dd bs=4K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 4096000000 bytes (4.1 GB) copied, 1.31167 seconds, 3.1 GB/s
$ dd bs=32K count=1000000 if=/dev/zero of=/dev/null 1000000+0 records in 1000000+0 records out 32768000000 bytes (33 GB) copied, 4.91775 seconds, 6.7 GB/s
see the difference?
IMHO it's a matter of what I call "block merge efficiency"....the more you stuff pages (that fits into a "magic" number), the faster I/O you got.
-- regards,
Mulyadi Santosa Freelance Linux trainer and consultant
blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev...
The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
On Wed, Jan 19, 2011 at 10:36 PM, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do, dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
However, when I do this with 2.6.37 I get close to 5GB/sec
Is there anything new in the kernel?
TIA
Mag, assuming you really meant 30 GB/sec vs. 5 GB/sec and there are no errors in your test, why do you care? ie. There is no comm or storage device I use that can run 5 GB/sec. In fact 1 GB/sec is pretty fast for most storage. Or is this more a curiosity thing? The performance loss could be /dev/zero, dd, or /dev/null. I would guess /dev/zero which is why I wonder what you need to track this down. A performance limited /dev/zero at 5 GB/sec is not a big issue for most uses. Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev... The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
This is just for curiosity. I think its strange this is occurring thats all. It did not happen in the older kernels but its happening on the newer versions. Possibly there maybe more like this. On Sun, Jan 23, 2011 at 10:31 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Wed, Jan 19, 2011 at 10:36 PM, Mag Gam <magawake@gmail.com> wrote:
Running on Redhat 5.1 if I do, dd bs=1024 count=1000000 if=/dev/zero of=/dev/null
I get around 30Gb/sec
However, when I do this with 2.6.37 I get close to 5GB/sec
Is there anything new in the kernel?
TIA
Mag,
assuming you really meant 30 GB/sec vs. 5 GB/sec and there are no errors in your test, why do you care?
ie. There is no comm or storage device I use that can run 5 GB/sec. In fact 1 GB/sec is pretty fast for most storage.
Or is this more a curiosity thing?
The performance loss could be /dev/zero, dd, or /dev/null. I would guess /dev/zero which is why I wonder what you need to track this down. A performance limited /dev/zero at 5 GB/sec is not a big issue for most uses.
Greg -- Greg Freemyer Head of EDD Tape Extraction and Processing team Litigation Triage Solutions Specialist http://www.linkedin.com/in/gregfreemyer CNN/TruTV Aired Forensic Imaging Demo - http://insession.blogs.cnn.com/2010/03/23/how-computer-evidence-gets-retriev...
The Norcross Group The Intersection of Evidence & Technology http://www.norcrossgroup.com
On Mon, Jan 24, 2011 at 7:28 AM, Mag Gam <magawake@gmail.com> wrote:
This is just for curiosity. I think its strange this is occurring that's all. It did not happen in the older kernels but its happening on the newer versions. Possibly there maybe more like this.
< snip of everything below the top post ;) For the future, we bottom post exclusively here. > I was not trying to shoot down your question, just find out if your were seeing a real world problem or not. The dd performance test you ran is basically artificial, so even if you track down why it slowed down, it could easily be that it is related to a functional improvement that the kernel team will consider much more important than an artificial performance test. A even more frustrating example is around 2.6.32 I think, the mdraid layer of the kernel got barrier support. Lot's of real world apps saw their performance drop significantly. But it was only because not having barriers is a lot faster than having them, but it is also inherently much more dangerous. And there is a mount option to disable barrier support. So users that complained were told it was their option. Either be safe and accept the new slow performance, or be risky and start using the nobarriers mount option. Only after track down the source of performance drop will you be able to have that discussion. Good luck hunting. fyi: How to perform that hunt is a very appropriate discussion for this list, but you might want to change the subject to reflect it if that's what you want to discuss next. Good Luck Greg
I appreciated the reply Greg! and thanks for the encouragement. On Mon, Jan 24, 2011 at 2:23 PM, Greg Freemyer <greg.freemyer@gmail.com> wrote:
On Mon, Jan 24, 2011 at 7:28 AM, Mag Gam <magawake@gmail.com> wrote:
This is just for curiosity. I think its strange this is occurring that's all. It did not happen in the older kernels but its happening on the newer versions. Possibly there maybe more like this.
< snip of everything below the top post ;) For the future, we bottom post exclusively here. >
I was not trying to shoot down your question, just find out if your were seeing a real world problem or not.
The dd performance test you ran is basically artificial, so even if you track down why it slowed down, it could easily be that it is related to a functional improvement that the kernel team will consider much more important than an artificial performance test.
A even more frustrating example is around 2.6.32 I think, the mdraid layer of the kernel got barrier support. Lot's of real world apps saw their performance drop significantly.
But it was only because not having barriers is a lot faster than having them, but it is also inherently much more dangerous. And there is a mount option to disable barrier support. So users that complained were told it was their option. Either be safe and accept the new slow performance, or be risky and start using the nobarriers mount option.
Only after track down the source of performance drop will you be able to have that discussion. Good luck hunting.
fyi: How to perform that hunt is a very appropriate discussion for this list, but you might want to change the subject to reflect it if that's what you want to discuss next.
Good Luck Greg
participants (3)
-
Greg Freemyer -
Mag Gam -
Mulyadi Santosa