(sort of a combo of ubuntu, systemtap and kernel question, hoping it's not too far off-topic.) for the purpose of an upcoming tutorial, i decided to install systemtap on my fully-updated 64-bit ubuntu 11.10 laptop, then build a new kernel and show how simple it is to use systemtap. it didn't turn out well. since i would be using a self-built kernel, i started here: http://sourceware.org/systemtap/wiki/SystemTapWithSelfBuiltKernel but i'm not convinced that page is entirely correct. while i made sure all of the listed kernel options were selected, my first concern is that i'm not sure why i should have to do "make headers_install". all that does is populate the in-tree usr/include/ directory with the exported header files. i'm not sure why that's necessary. is it really required? does systemtap actually go looking there for header files? also, the rest of that page seems to explain how to make sure systemtap can find your kernel build tree. but if you build and install, then /lib/modules/<release>/build is a symlink to the build tree, so i'm not sure anything else has to be done. the rest of that page also seems unnecessary. thoughts? in any event, i configured, built and installed my new kernel (based on latest "git pull" from development kernel source), and rebooted to get: $ uname -r 3.4.0-rc2+ $ which is correct so, theoretically, i should be read to try a simple systemtap example. reading here: http://www.domaigne.com/blog/random/running-systemtap-on-ubuntu/ i tried: $ stap -e 'probe kernel.function("sys_open") {log("hello world") exit()}' and got: ===== start ===== In file included from /usr/share/systemtap/runtime/transport/transport.c:53:0, from /usr/share/systemtap/runtime/print.c:18, from /usr/share/systemtap/runtime/runtime.h:128, from /tmp/stapadEYdA/stap_59bddd394f89d5f186da7e250db36434_876.c:55: /usr/share/systemtap/runtime/transport/relay_v2.c:242:2: error: initialization from incompatible pointer type [-Werror] /usr/share/systemtap/runtime/transport/relay_v2.c:242:2: error: (near initialization for ‘__stp_relay_callbacks.create_buf_file’) [-Werror] In file included from /usr/share/systemtap/runtime/print.c:18:0, from /usr/share/systemtap/runtime/runtime.h:128, from /tmp/stapadEYdA/stap_59bddd394f89d5f186da7e250db36434_876.c:55: /usr/share/systemtap/runtime/transport/transport.c: In function ‘_stp_get_root_dir’: /usr/share/systemtap/runtime/transport/transport.c:380:8: error: ‘struct hlist_head’ has no member named ‘next’ In file included from /tmp/stapadEYdA/stap_59bddd394f89d5f186da7e250db36434_876.c:57:0: /usr/share/systemtap/runtime/stat.c: In function ‘_stp_stat_get’: /usr/share/systemtap/runtime/stat.c:213:2: error: ‘cpu_possible_map’ undeclared (first use in this function) /usr/share/systemtap/runtime/stat.c:213:2: note: each undeclared identifier is reported only once for each function it appears in /usr/share/systemtap/runtime/stat.c: In function ‘_stp_stat_clear’: /usr/share/systemtap/runtime/stat.c:248:2: error: ‘cpu_possible_map’ undeclared (first use in this function) cc1: all warnings being treated as errors make[1]: *** [/tmp/stapadEYdA/stap_59bddd394f89d5f186da7e250db36434_876.o] Error 1 make: *** [_module_/tmp/stapadEYdA] Error 2 Pass 4: compilation failed. Try again with another '--vp 0001' option. ===== end ===== that doesn't look good. in particular, this error just looks strange: /usr/share/systemtap/runtime/transport/transport.c:380:8: error: ‘struct hlist_head’ has no member named ‘next’ to make a long story short, that diagnostic is entirely correct -- "struct hlist_head" as it's defined in <types.h> is: struct hlist_head { struct hlist_node *first; }; but that systemtap source file is trying to refer to a member called "next", which clearly does not exist. am i doing something silly? can someone else with an ubuntu system try this and let me know what happens? thanks. rday -- ======================================================================== Robert P. J. Day Ottawa, Ontario, CANADA http://crashcourse.ca Twitter: http://twitter.com/rpjday LinkedIn: http://ca.linkedin.com/in/rpjday ========================================================================