Skip to content

Tracing

While an application generated by Code RealTime is running, you can turn on tracing of messages that are exchanged between capsule instances at run-time. Such traces can be visualized graphically as sequence diagrams and can be very useful when troubleshooting certain kinds of bugs in an application such as race conditions, dead-locks, performance problems, unexpected error situations etc. Here are some examples of questions that can be answered by analyzing captured traces:

  • Which messages were received by a capsule that caused it to enter a certain state (e.g. an unexpected error state)?
  • Why did a capsule instance receive a certain message (e.g. an unexpected error message)?
  • Which capsule instances in the application exchange messages most frequently? (For performance reasons you typically want such capsule instances to run in the same thread since intra-thread communication is much faster than inter-thread communication.)
  • Are there capsule instances in the application which seem to be unused (and hence could be candidates for removal)?

Trace File Formats

Code RealTime supports capturing traces in two different file formats:

  1. .art-trace This is Code RealTime's own trace format which is supported by the tool itself. If you open an .art-trace file in Code RealTime you benefit from features such as syntax coloring, navigation etc. You can show an .art-trace file graphically in a sequence diagram viewer which is optimized for handling big trace files directly inside Code RealTime. To open a sequence diagram right-click on the .art-trace file in the Explorer view, or in the text editor, and perform the command Open Sequence Diagram.
  2. .ms This trace format is understood by the js-sequence-diagrams web service which can render the trace as a sequence diagram on a web page. These sequence diagrams contain less details than .art-trace sequence diagrams and are mainly intended to serve as documentation that can explain how an application behaves internally in certain situations.

.art-trace

The image below shows a small .art-trace file which has been captured from the TcpRangeCounter sample. The sequence diagram shows so called lifelines a the top to represent the capsule instances in the application. We can see lifelines that represent the counter and server capsule instances and we can see that the server sent a message for the setMax event with 12 as data to the counter. It then sent also the resumeCounting event (without any data).

The lifeline for a capsule instance has the name of the part that contains the instance. If the part has non-single multiplicity, the lifeline shows the index of the capsule instance within square brackets ([0] for the first instance). If you hover over a lifeline a tooltip appears with more information, such as the dynamic type of the capsule instance and its structured path in the application's composite structure.

Colors are used to distinguish three types of lifelines:

  • Green for the lifeline that represents the top capsule instance which is created at application start-up. It has the name "application".
  • Dark blue lifelines represent capsule instances that are created after the top capsule instance has been created, when the application runs.
  • Light blue lifelines represent certain TargetRTS components that may interact with capsule instances.

There are two TargetRTS components that can interact with capsule instances. <timer> is the component of the TargetRTS that implements timers and you will see when a capsule instance receives a timeout message for a timer that has timed out. Inside the TargetRTS this capsule instance has the name specials and is typed by RTTimerActor. The <system> lifeline is used for showing when capsule instances are created and initialized (by sending the initialize message).

The sequence diagram may contain note boxes that show if the tracing was paused for some time. This indicates that the sequence diagram doesn't show messages that may have been exchanged during that time.

.ms

The image below shows the same trace as above but this time captured in the .ms file format. The js-sequence-diagrams web page is used for rendering the sequence diagram.

Note that this diagram contains less details than an .art-trace sequence diagram. For example, there are no colors and no tooltip for showing the structure path of a capsule instance. Another difference is that the note box that shows tracing was paused only appears on one lifeline.

Turning Tracing On and Off

By default the tracing capability is enabled in the TargetRTS. You can disable it through the RTS_TRACE configuration setting if you don't plan to capture any traces. This will make your executable slightly smaller.

When your application starts to run, tracing is initially turned off. To start capturing a trace you need to turn it on. There are several ways to do this as described in the sections below.

While tracing is turned on, trace events will be captured and written to the trace file. However, for performance reasons the file is not immediately flushed. The trace file will be flushed when the application terminates, but quite often you may want to do it before that. Refer to the sections below for how to force the trace file to be flushed.

Captured trace files will be placed in the same folder as where the running executable is located. By default traces in the .art-trace file format will be captured. To use the .ms trace file format you need to start the application with an extra command-line argument -traceFormat=ms.

Capturing all messages that are exchanged in an application for the whole time it runs will usually lead to very large traces. Such big traces are cumbersome to work with, and sequence diagrams may not be possible to render. And even if they could be rendered, they may not be easy to use due to the size. It's better to only turn on tracing when the application reaches a point when something interesting is about to happen. For example, if you are troubleshooting a bug that occurs when a certain transition runs in a capsule's state machine, you can debug the application with the Art Debugger and turn on tracing when the source state of that transition is reached. Then you can manually send the event which will trigger the interesting transition, and then turn tracing off. Thereby the trace file will only show what happened when that transition executed, which makes it small and manageable.

Or you could debug your application with a C++ debugger and put a breakpoint that gets hit before something interesting happens. Then you can turn on tracing and let the application run until it hits another breakpoint, and then turn off tracing again. The trace then shows you which messages were exchanged in the application between those two breakpoints were hit.

From the Command Line

If you want to trace what happens in the application from the very beginning you can turn tracing on from the command-line when you launch the application. Launch your executable with the -URTS_DEBUG argument as the very first argument. For example:

./Top.EXE -URTS_DEBUG="trace on;quit"

This command will turn on tracing at application start-up and then quit the RTS Debugger to let the application run without debugger at full speed. In this case the only way to turn off tracing before the application terminates is to do it programmatically using TargetRTS APIs.

You can also launch the application like this:

./Top.EXE -URTS_DEBUG="trace on;continue" -obslisten=12345

The main difference here is that you then later can attach the Art Debugger on the specified port, and turn off tracing from there.

Other RTS Debugger commands can of course also be used. For example, to trace the first 5 messages that are sent in the application:

./Top.EXE -URTS_DEBUG="trace on;go 5"

Read more about the -URTS_DEBUG argument here.

Note that the non-default .ms trace file format can only be turned on from the command-line, by means of the -traceFormat=ms command-line argument. The trace file format cannot be changed while the application is running.

From the RTS Debugger

The RTS Debugger provides a command trace for turning tracing on or off:

RTS debug: ->trace on
  Tracing is on

If you are unsure whether tracing is currently on or off, you can run the command without argument.

A useful side-effect of the trace command is that it will flush the current trace file, if any.

From the Art Debugger

To turn tracing on or off from the Art Debugger invoke the Send to App command from the command palette. The trace command from the RTS Debugger can be accessed from the popup that appears.

From the C++ Debugger

Turning tracing on or off when you debug your application in a C++ debugger is often very useful. You do it by using the API provided by the TargetRTS class RTTracer:

  • RTTracer::setEnabled(bool) Turns tracing on or off.
  • RTTracer::isEnabled() Returns if tracing is currently turned on or off.
  • RTTracer::flushTrace() Forces the current trace file, if any, to be flushed, so it contains all trace events received so far.

Exactly how to call these functions depends on which C++ debugger you are using. For example, in the Visual Studio C++ Debugger you do it from the Immediate Window:

A typical workflow may look like this:

  1. Build the application and the TargetRTS with debug symbols as described here.
  2. Place a breakpoint at a place where you want to start tracing.
  3. Start the application from the C++ debugger and let it run until the breakpoint is hit.
  4. Call RTTracer::setEnabled(true) from the C++ debugger to turn tracing on.
  5. Place a breakpoint at a place where you want to stop tracing.
  6. Resume execution of the application and let it run until the breakpoint is hit.
  7. Call RTTracer::setEnabled(false) from the C++ debugger to turn tracing off. Or call RTTracer::flushTrace() if you want to keep tracing but want to check what the trace contains so far.

Programmatically

You can turn tracing on or off from your application code by using the API provided by the TargetRTS class RTTracer. See above for which functions that are available to use.

One scenario where it can be useful to enable tracing programmatically is if your C++ debugger does not support calling functions, or if you need to trace a certain scenario in an application that doesn't have debug symbols. It can also be useful to build tracing functionality into your application and for example use it to trace unexpected error scenarios that may occur. The collected trace files can then help you troubleshoot problems that have occurred when the application ran in the production environment.