Is it possible to use a local drive to cache an NFS share
Hi All, I am mounting /home over NFS which is great, but it really kills compile times. So I have a local HDD which I have copied all my source code over from which I do my coding and compiling. What I would love to do is use the local drive to cache /home. Specifically one user directory under /home - only one machine will ever be modifying the contents of this directory. Or should I just use rsync? On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an appreciable increase in compile speed. Regards, Graeme
Hi! On 23:00 Mon 03 Sep , Graeme Russ wrote:
Hi All,
I am mounting /home over NFS which is great, but it really kills compile times. So I have a local HDD which I have copied all my source code over from which I do my coding and compiling.
What I would love to do is use the local drive to cache /home. Specifically one user directory under /home - only one machine will ever be modifying the contents of this directory.
NFS is not really made for caching. Its design idea was mainly that the server should be stateless. There is basically no way to get a notification whether a file was changed remotely without either polling or running an external daemon. There are 2 network file systems which support client caching: codafs and andrewfs. However, they are rather complex to setup and I guess nobody is really using them. If nobody else changes the files you should be fine. But I still would not recommend doing this, because this is basically a time bomb which might go off after you forget what you are doing now. Also there should not be that much performance difference for your worklead unless either your network or your nas is very slow. After a quick search I have found this project (not sure what it does exactly): http://sourceforge.net/apps/mediawiki/fuse/index.php?title=FileSystems#mcach...
Or should I just use rsync?
How do you want to sync it back?
On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an appreciable increase in compile speed.
This does not suprise me. Compiling mostly stresses the cpu(s). -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
Hi :) On Mon, Sep 3, 2012 at 8:00 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
Hi All,
I am mounting /home over NFS which is great, but it really kills compile times. So I have a local HDD which I have copied all my source code over from which I do my coding and compiling.
What I would love to do is use the local drive to cache /home. Specifically one user directory under /home - only one machine will ever be modifying the contents of this directory.
I suggest to use ccache, if your focus is only about compiling. Then configure ccache to use your local drive as the directory to store the cache objects. IMHO, that should be doable :) -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
Hi! On 22:27 Mon 03 Sep , Mulyadi Santosa wrote:
Hi :)
On Mon, Sep 3, 2012 at 8:00 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
Hi All,
I am mounting /home over NFS which is great, but it really kills compile times. So I have a local HDD which I have copied all my source code over from which I do my coding and compiling.
What I would love to do is use the local drive to cache /home. Specifically one user directory under /home - only one machine will ever be modifying the contents of this directory.
I suggest to use ccache, if your focus is only about compiling.
Then configure ccache to use your local drive as the directory to store the cache objects.
ccache caches the compilation result, but it still needs to reads the source code files and write the object files... -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
Hi, NFS client support FSCache support which is default in recent kernels CONFIG_NFS_FSCACHE=y, But as I remember, it is a read cache (not even write-through), so a compile load generates lots of new files, so those won't really be benefited by FSCache. But as you are pointing out, your setup is a single user access, so probably you can relax some of the NFS consistency options, e.g. async mount is really suitable for your case, also you can increase actimeo values. Compiler writes will hit-page cache and with asynchronous writebacks, it is way more faster than SSDs :). -Rajat On Mon, Sep 3, 2012 at 6:30 PM, Graeme Russ <graeme.russ@gmail.com> wrote:
Hi All,
I am mounting /home over NFS which is great, but it really kills compile times. So I have a local HDD which I have copied all my source code over from which I do my coding and compiling.
What I would love to do is use the local drive to cache /home. Specifically one user directory under /home - only one machine will ever be modifying the contents of this directory.
Or should I just use rsync?
On a side note - I moved my source code from a 1TB HDD on a 3Gb/s SATA port to an Intel 510 series 250GB SSD on a 6Gb/s SATA port. But I didn't see an appreciable increase in compile speed.
Regards,
Graeme
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
participants (4)
-
Graeme Russ -
michi1＠michaelblizek.twilightparadox.com -
Mulyadi Santosa -
Rajat Sharma