DTrace Topics Guide
From Siwiki
Contents |
[edit] DTrace Topics: Guide
This article is a guide to programming in DTrace, and is part of the DTrace Topics collection. A general understanding of DTrace is assumed knowledge, which can be studied from the DTrace Topics: Intro section. DTrace is a dynamic troubleshooting and analysis tool first introduced in the Solaris 10 and OpenSolaris operating systems.
Completion 
Difficulty 
Audience All DTrace users
[edit] DTrace Guide
This DTrace Guide focuses on summaries, examples and suggested usage.
For a reference on DTrace, see the official docs.sun.com DTrace Guide which is also an excellent source of examples.
[edit] Privileges
Only root or users with DTrace privileges can run DTrace. If an unprivileged user trys to run /usr/sbin/dtrace,
$ id uid=1001(user1) gid=1(other) $ /usr/sbin/dtrace -n 'syscall::exece:return' dtrace: failed to initialize dtrace: DTrace requires additional privileges
Non-root users need one or more of the following privileges to use DTrace:
| Privilege Name | Description |
| dtrace_user | to examine their syscalls, and to profile their own processes |
| dtrace_proc | to trace their own processes |
| dtrace_kernel | to examine the kernel. Danger! |
The dtrace_kernel privilege is dangerous as it allows the user read only access across the kernel. With some DTrace expertise, any data that the server acts upon could be viewed, including plaintext passwords and files. Fortunately, the dtrace_user and dtrace_proc privileges may be sufficient for many users of DTrace, especially developers who wish to DTrace their own applications.
[edit] Adding Privileges
Interfaces for privilege control include ppriv(1), usermod(1M) and user_attr(4). Adding DTrace privileges to the user1 account can be performed in the following way:
As user1,
$ id
uid=1001(user1) gid=1(other)
$ ppriv $$
214885: -sh
flags = <none>
E: basic
I: basic
P: basic
L: all
$ exit
As root,
# grep user1 /etc/user_attr
# usermod -K defaultpriv=basic,dtrace_user,dtrace_proc user1
# grep user1 /etc/user_attr
user1::::type=normal;defaultpriv=basic,dtrace_user,dtrace_proc
# su - user1
$ ppriv $$
285007: -sh
flags = <none>
E: basic,dtrace_proc,dtrace_user
I: basic,dtrace_proc,dtrace_user
P: basic,dtrace_proc,dtrace_user
L: all
The user1 account begins as an unprivileged user (UID != 0 and no entry in /etc/user_attr). root then runs usermod to add two DTrace privileges to the user1 account. This puts an entry in /etc/user_attr.
[edit] Probes
Probes are the instrumentation point from which DTrace can collect data. Their names are a four-tuple: provider:module:function:name,
| Provider | A library of related probes. |
| Module | The module the function belongs to, either a kernel module or user segment. |
| Function | The function name that contains the probe. |
| Name | The name of the probe. |
At the command line,
- Listing probes: dtrace -ln probename
- Tracing probes: dtrace -n probename
[edit] Specifing Probe Names
A blank field in a probe name is a wildcard, as is '*'. The following are examples of this:
- syscall:::
- List everything from the syscall provider.
- syscall::open*:entry
- List all syscall entry probes when the function begins with “open”. This matches open and open64.
The following uses -ln to list the matched probe names:
# dtrace -ln 'syscall::open*:entry' ID PROVIDER MODULE FUNCTION NAME 75947 syscall open entry 76335 syscall open64 entry
[edit] Understanding Probe Names
The best way is to take the provider name, then look up the related chapter in the DTrace Guide (docs.sun.com).
Examples of probe name interpretation:
- io:genunix:bdev_strategy:start
- The start probe in the io provider, for bdev_strategy() in module genunix. (DTrace Guide, chapter 27, io).
- dtrace:::BEGIN
- The BEGIN probe in the dtrace provider. (DTrace Guide, chapter 17, dtrace).
[edit] Finding Probes
With thousands of probes available, it can take some time to find which probes are relevant.
You may get some mileage from grep(1) and regular expressions. Eg, what traces kill(1) activity?:
# dtrace -l | grep kill
[...42 lines truncated...]
63725 fbt ufs ufsfx_kill_fix_failure_thread entry
63726 fbt ufs ufsfx_kill_fix_failure_thread return
63930 fbt ufs logmap_kill_roll entry
63931 fbt ufs logmap_kill_roll return
71717 fbt lx_brand lx_kill entry
71718 fbt lx_brand lx_kill return
71799 fbt lx_brand lx_tkill entry
71800 fbt lx_brand lx_tkill return
75527 fbt md_mirror resync_kill_pending entry
75528 fbt md_mirror resync_kill_pending return
76011 syscall kill entry
76012 syscall kill return
76221 syscall lwp_kill entry
76222 syscall lwp_kill return
# dtrace -l | grep kill | wc -l
56
This approach helped (in this case), filtering ~70,000 probes down to 56 potentials.
[edit] Finding Probes Checklist
Approaches for finding probes,
- Use dtrace -ln with '*', or dtrace -l with grep(1).
- Run your workload, watch what fires.
- if it doesn't fire, it is almost certainly not related.
- Run a known workload with a count of n (where n is a moderately sized prime), see what fires by a factor of n.
- Eg, 23 database transactions.
- Think of triggered events, and study their stack traces.
- Eg, the stacks for io:::start shows who caused I/O.
- Read the source, if available.
- Ask on dtrace-discuss@opensolaris.org.
