> 2GB memset is very slow

[Jason Nymble]

On 01 Feb 2011, at 2:48 PM, vinit dhatrak wrote:

> 
> 
> On Tue, Feb 1, 2011 at 3:53 PM, Jason Nymble <jason.nymble at gmail.com> wrote:
> I was using memset on a reserved area of memory (64bit x86 kernel and system), and noticed that as soon as I exceed a size of 2GB, the function becomes extremely slow, e.g just below 2GB it takes typically about 0.3s, and just above 2GB is takes about 39s to complete...
> 
> I tried tracing the eventual function that is called in the kernel, and I think it resolves to the below (even on x86_64 if I'm not mistaken):
> static inline void *__memset_generic(void *s, char c, size_t count)
> {
>        int d0, d1;
>        asm volatile("rep\n\t"
>                     "stosb"
>                     : "=&c" (d0), "=&D" (d1)
>                     : "a" (c), "1" (s), "0" (count)
>                     : "memory");
>        return s;
> }
> 
> size_t is defined as (unsigned long) on my platform, but I suspect the d0 and d1 variables above cause problems because they are int... Is this a kernel bug, or known limitation, or what?
> _______________________________________________
> Kernelnewbies mailing list
> Kernelnewbies at kernelnewbies.org
> http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
> 
> 
> Hi Jason,
> 
> How did you allocate the memory? What is the size of physical memory that is attached to your machine ?
> 
> 

The machine has 48GB of physical memory, and I reserve a chunk as a kernel param with the memmap=nn[KMG]$ss[KMG] syntax (see Documentation/kernel-parameters.txt), e.g. memmap=8G$4G to reserve an 8GB chunk of mem starting at offset 4GB. Then in the driver I simply use ioremap_cache() with the physical base offset and size I require to get a kernel virtual address. If I do that across any 4GB physical boundary, then access to that memory via that kernel virtual address pointer is extremely slow.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110201/131e1f87/attachment-0001.html