MAX limit of file descriptor
hi:) In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ? In network programing ,what is the essential for the maximum of connections dealed per second thanks!
i can only make a general statement, may not be always true/false: in the kernel almost EVERYTHING HAS TO BE FINITE....and this is cater for the fact that but at the userspace or application level, u can design structures to be infinite. eg, I used python for large number calculation, and so far it has not limits, but I am sure at the representation level, there is one....but because i don't know the datastructure used, i don't know the limits. On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
hi:)
In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ?
In network programing ,what is the essential for the maximum of connections dealed per second
thanks!
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
I believe you can achieve the 'effect' of setting the maximum number of open file descriptors to 'infinite' by using the following (untested code!): <snip> #include <sys/time.h> #include <sys/resource.h> struct rlimit rlim; rlim.rlim_cur = RLIM_INFINTY; rlim.rlim_max = RLIM_INFINITY; int ret = setrlimit(RLIMIT_NOFILE, &rlim); </snip> Note that what Peter pointed out is more or less correct. You can't *actually *have infinite file descriptors. The above code just ensures that the kernel won't perform any checks on the number of open file descriptors. So, if you hit a real resource limit; like no more memory left for opening files, you would still get an error. My feeling is that this isn't a wise thing to do. It's always better to know a precise number of the upper limit rather than an arbitrarily large number. ~ Gaurav On Sat, Feb 9, 2013 at 4:49 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
i can only make a general statement, may not be always true/false:
in the kernel almost EVERYTHING HAS TO BE FINITE....and this is cater for the fact that
but at the userspace or application level, u can design structures to be infinite. eg, I used python for large number calculation, and so far it has not limits, but I am sure at the representation level, there is one....but because i don't know the datastructure used, i don't know the limits.
On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
hi:)
In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ?
In network programing ,what is the essential for the maximum of connections dealed per second
thanks!
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Gaurav Jain Associate Software Engineer VxVM Escalations Team, SAMG Symantec Software India Pvt. Ltd.
thanks! Actually , my question comes from network performance ,I want to know ,in per second ,the maximum of tcp connections that can be dealed with by my server. How can I do the test and calculate the connection number , Is it possible that my server can deal with 10k tcp connections per second? what is the relationship between this and throughput rate? Is there document that tells the best optimization of this ?
On Mon, Feb 11, 2013 at 5:07 AM, horseriver <horserivers@gmail.com> wrote:
thanks!
Actually , my question comes from network performance ,I want to know ,in per second ,the maximum of tcp connections that can be dealed with by my server.
AFAIK, the only way to find out is by doing benchmark by your own. You can use tools like netperf for that purpose. -- regards, Mulyadi Santosa Freelance Linux trainer and consultant blog: the-hydra.blogspot.com training: mulyaditraining.blogspot.com
On Mon, 11 Feb 2013 06:07:38 +0800, horseriver said:
Actually , my question comes from network performance ,I want to know ,in per second ,the maximum of tcp connections that can be dealed with by my server.
That will be *highly* dependent on what your server code does with each connection. A "hello world" reply and close socket will, of course, go lots faster than something that has to go contact an enterprise-scale database, do 3 SQL joins, and format the results.
How can I do the test and calculate the connection number , Is it possible that my server can deal with 10k tcp connections per second?
10K/sec peaks can be achieved even on a laptop, assuming a dummy do-nothing service. Keeping that sustained for a real application will depend on the service time needed - if you have 20 CPUs in the box, and spread the load across all 20, you have to average under 2ms to service each request, which will be a killer if you have to go to disk at all for a request. At that point, the guys at Foundry will be more than happy to sell you a load-balancer so you can have a stack of 10 20-CPU servers each of which only handles 1K/sec and thus has a 20ms time budget.
what is the relationship between this and throughput rate?
Lots of tiny connections will totally suck at aggregate throughput, if for no other reason than TCP slow-start never gets a chance to really open the transmit window up. But in general, there is always a trade-off between transaction rate and throughput.
Is there document that tells the best optimization of this ?
"best" is defined by what your application actually needs. The "best" settings for my NFS server will be totally different than what the HTTP server 12 racks over needs...
Hi! On 13:10 Sat 09 Feb , horseriver wrote:
hi:)
In one process ,what is the max number of opening file descriptor ?
Type "ulimit -a" in your shell. On my system (debian) the default is 1024.
Can it be set to infinite ?
Maybe, but at least it can be set very high.
In network programing ,what is the essential for the maximum of connections dealed per second
- Use non blocking i/o and epoll(). Do *not* create 1 process/thread for each connection and do not use use select(). - Obviously, the more memory your application uses, the more memory has to be put in the server. IIRC, 1 tcp connection uses ~1kb kernel memory. - The same applies for cpu time. On the system side, you may want to recommend network adaptors which can be switched to polling instead of raising 1 interrupt per packet. You should expect to see lots of small packets on the network. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
On Sun, Feb 10, 2013 at 8:29 PM, <michi1@michaelblizek.twilightparadox.com>wrote:
Hi!
On 13:10 Sat 09 Feb , horseriver wrote:
hi:)
In one process ,what is the max number of opening file descriptor ?
Type "ulimit -a" in your shell. On my system (debian) the default is 1024.
Hi Michael, nice to see u again. BTW, many of the parameters as reported by ulimit, also has to be taken with some doubts: ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 47543 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 47543 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited the above is for Ubuntu 12.04 with 32-bit kernel (3.2.0) but of course we know that max file size has a limit - depending on whether it is ext2 or ext3 or ext4. cannot remember the exact nos, but general conceptual level, there is a limit. even for "CPU time"...it is limited by the underlying bit length of representation for time. as usual...i don't know the details :-(, just concept. sorry :-(.
Can it be set to infinite ?
Maybe, but at least it can be set very high.
In network programing ,what is the essential for the maximum of connections dealed per second
- Use non blocking i/o and epoll(). Do *not* create 1 process/thread for each connection and do not use use select(). - Obviously, the more memory your application uses, the more memory has to be put in the server. IIRC, 1 tcp connection uses ~1kb kernel memory. - The same applies for cpu time. On the system side, you may want to recommend network adaptors which can be switched to polling instead of raising 1 interrupt per packet. You should expect to see lots of small packets on the network.
-Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
Hi! On 22:47 Sun 10 Feb , Peter Teoh wrote:
On Sun, Feb 10, 2013 at 8:29 PM, <michi1@michaelblizek.twilightparadox.com>wrote:
Hi!
On 13:10 Sat 09 Feb , horseriver wrote:
hi:)
In one process ,what is the max number of opening file descriptor ?
Type "ulimit -a" in your shell. On my system (debian) the default is 1024.
Hi Michael, nice to see u again.
BTW, many of the parameters as reported by ulimit, also has to be taken with some doubts:
ulimit -a core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 47543 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 8192 cpu time (seconds, -t) unlimited max user processes (-u) 47543 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
the above is for Ubuntu 12.04 with 32-bit kernel (3.2.0) but of course we know that max file size has a limit - depending on whether it is ext2 or ext3 or ext4. cannot remember the exact nos, but general conceptual level, there is a limit.
Well, ulimit is not really there to tell you what the system can handle. It only tries to limit the ressources which can be used for stability/security reasons. Also, even if ulimit would want to tell you what the system can handle, the interface is obviously not suited to export the file system limits when you have more than one file system mounted. File system limits: http://en.wikipedia.org/wiki/Ext3 http://en.wikipedia.org/wiki/Ext4
even for "CPU time"...it is limited by the underlying bit length of representation for time. as usual...i don't know the details :-(, just concept. sorry :-(.
ulimit -t 1 dd if=/dev/zero of=/dev/null It seems cpu limit means that the process will be killed if it exceeds it. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
On Sat, 09 Feb 2013 13:10:47 +0800, horseriver said:
In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ?
In network programing ,what is the essential for the maximum of connections dealed per second
In general, you'll find that "number of file descriptors" isn't what ends up killing you for high-performance network programming. What usually gets you are things like syn floods (either intentional ddos or getting slashdotted), because each time you do an accept() on an incoming connection you end up using userspace resources to handle the connection. So the *real* question becomes "how many times per second is your box able to fork() off an httpd, do all the processing required, and close the connection?" A secondary gotcha is that dying TCP connections end up stuck in FIN-WAIT and FIN-WAIT-2, And if you're trying to drive multiple 10G interfaces at line speed, it gets even more fun. Fortunately, for my application (high performance disk servers) the connections are mostly persistent, so it's "only" a problem of getting disks to move data that fast. :)
Hi! On 14:24 Mon 11 Feb , Valdis.Kletnieks@vt.edu wrote:
On Sat, 09 Feb 2013 13:10:47 +0800, horseriver said: ...
In network programing ,what is the essential for the maximum of connections dealed per second ... So the *real* question becomes "how many times per second is your box able to fork() off an httpd, do all the processing required, and close the connection?"
If you fork(), than this definitely will be your bottleneck (except the application does something even slower). Better use epoll+nonblocking i/o. -Michi -- programing a layer 3+4 network protocol for mesh networks see http://michaelblizek.twilightparadox.com
perhaps i can add more info, after doing more investigation: a. "ulimit" is a shell feature, it is not a command line binary. "man bash" and "man sh" and u can see ulimit has different feature available for u. b. ulimit control all the resources defined by the processes spawn from the current shell onwards...ie, once ulimit is change, all child processes from that shell onwards will change. but resources limit in another shell, existing processes etc does not. c. ulimit is a userspace feature, the kernel will have all the corresponding feature of max open files etc...but definitely it is not unlimited like that of ulimit. d. to see ALL the open files u can use "lsof" and "-p" give u control to point at which process to dig for open files. it also list all the open connections (TCP) for u....which is what u want. e. generally java applications will open many many files descriptor concurrently: http://www.java.net/forum/topic/glassfish/glassfish/too-many-open-files-issu... (above listed 4500, and many others java apps like IBM RSA also have many). On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
hi:)
In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ?
In network programing ,what is the essential for the maximum of connections dealed per second
thanks!
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
one more: To modify system-wide limits: */etc/security/limits.conf* On Tue, Feb 12, 2013 at 5:31 PM, Peter Teoh <htmldeveloper@gmail.com> wrote:
perhaps i can add more info, after doing more investigation:
a. "ulimit" is a shell feature, it is not a command line binary. "man bash" and "man sh" and u can see ulimit has different feature available for u.
b. ulimit control all the resources defined by the processes spawn from the current shell onwards...ie, once ulimit is change, all child processes from that shell onwards will change. but resources limit in another shell, existing processes etc does not.
c. ulimit is a userspace feature, the kernel will have all the corresponding feature of max open files etc...but definitely it is not unlimited like that of ulimit.
d. to see ALL the open files u can use "lsof" and "-p" give u control to point at which process to dig for open files. it also list all the open connections (TCP) for u....which is what u want.
e. generally java applications will open many many files descriptor concurrently:
http://www.java.net/forum/topic/glassfish/glassfish/too-many-open-files-issu...
(above listed 4500, and many others java apps like IBM RSA also have many).
On Sat, Feb 9, 2013 at 1:10 PM, horseriver <horserivers@gmail.com> wrote:
hi:)
In one process ,what is the max number of opening file descriptor ? Can it be set to infinite ?
In network programing ,what is the essential for the maximum of connections dealed per second
thanks!
_______________________________________________ Kernelnewbies mailing list Kernelnewbies@kernelnewbies.org http://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies
-- Regards, Peter Teoh
-- Regards, Peter Teoh
participants (6)
-
Gaurav Jain -
horseriver -
michi1＠michaelblizek.twilightparadox.com -
Mulyadi Santosa -
Peter Teoh -
Valdis.Kletnieks＠vt.edu