Page tree
Skip to end of metadata
Go to start of metadata

Configuring and Enabling MDC (Mapped Diagnostic Context) logging to log contextual messages. Enabling MDC can provide additional information on which Camel Context and the specific Route that created a log message. This can be very useful when you have multiple camel contexts creating logs at the same time.

The following two log entries are from two separate camel contexts, camelContext id="CTIngestCamelContext" and camelContext id="DerivativesCamelContext"  from camera-trap-route.xml and derivatives-route.xml respectively that are generating logs at the same time.

Without MDC logging enabled.

Without MDC logging
2016-03-18 15:34:28,299 | INFO  | pool-43-thread-1 | DerivativesStartProcessing       | 136 - org.apache.camel.camel-core - 2.16.1 | ID:jbirkhimer-dev.net-54729-1458142567258-4:1:2:2:749 Derivatives: No message processing required.

2016-03-18 15:34:34,310 | INFO  | pool-43-thread-1 | ctingest                         | 136 - org.apache.camel.camel-core - 2.16.1 | ID-jbirkhimer-dev-net-37584-1458321773228-14-87 CameraTrapIngest: Starting Post validation ...

With MDC logging enabled and configured to display the ContextId and RouteId

With MDC logging enabled.
2016-03-18 15:46:15,272 | INFO  | pool-55-thread-1 | DerivativesStartProcessing       | 136 - org.apache.camel.camel-core - 2.16.1 | ContextID: DerivativesCamelContext | RouteId: DerivativesStartProcessing | ID:jbirkhimer-dev.net-37549-1458330022891-4:1:2:1:133 Derivatives: No message processing required.

2016-03-18 15:46:21,288 | INFO  | pool-55-thread-1 | ctingest                         | 136 - org.apache.camel.camel-core - 2.16.1 | ContextID: CTIngestCamelContext | RouteId: CameraTrapValidateIngest | ID-jbirkhimer-dev-net-37584-1458321773228-20-87 CameraTrapIngest: Starting Post validation ...

As you can see with MDC logging enabled you are able to see exactly which context and route generated the log message. Now imagine you have several other camel contexts generating logs in addition to the two above at the same time.

BONUS:

We can take MDC logging a step further and create a separate log file for each context and even down to each route using MDC logging. See below for configuring log files using MDC logging.

 

Camel MDC logging

Enabling MDC logging in Camel

  1. Java DSL

    Enable MDC logging for Camel Java DSL
    CamelContext context = ...
    context.setUseMDCLogging(true);
    ...
  2. XML DSL ( useMDCLogging="true" )

    Enable MDC logging for Camel XML DSL
    <camelContext xmlns="http://camel.apache.org/schema/blueprint" useMDCLogging="true">  ...
    </camelContext>
  3. log4j.properties for Camel MDC context logging can be usefull during unit testing and running camel from IDE.

    log4j.properties
    log4j.appender.stdout.layout.ConversionPattern=[%-5p] [%c{1}.%M():%L] | Camel ContextID: %X{camel.contextId} | Camel RouteId: %X{camel.routeId} | %m%n
  4. Camel 2.10 onwords provides the following context information available for MDC logging (ref: http://camel.apache.org/mdc-logging.html )

    KeyDiscription
    camel.exchangeIdThe exchange id
    camel.messageIdThe message id
    camel.correlationIdThe correlation id of the exchange if it's correlated. For example a sub message from the Splitter EIP
    camel.transactionKeyThe id of the transaction for transacted exchanges. Note the id is not unique, but its the id of the transaction template that marks the transaction boundary for the given transaction. Hence we decided to name the key transactionKey and not transactionID to point out this fact.
    camel.routeIdThe id of the route, in which the exchange is currently being routed
    camel.breadcrumbIdAn unique id used for tracking messages across transports.
    camel.contextId

    The camel context id used for tracking the message from different camel context.

Servicemix / Karaf MDC logging

Enabling MDC logging in Servicemix / Karaf. 

  1. Add %X{camle.contextId} and %X{camel.routeId} to the log4j layout.ConversionPattern for each log4j appender in the following two servicemix config. files.

    org.ops4j.pax.logging.cfg
    ...
    # CONSOLE appender not used by default
    log4j.appender.stout.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n
    ...
    # File appender
    log4j.appender.out.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n
    ...
    # Sift appender
    log4j.appender.sift.layout.ConversionPattern=%d | %-5p | %t | %c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n
    ...
    org.apache.karaf.log.cfg
    ...
    pattern = %d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n

Bonus: Taking MDC logging a step further and create a separate log file for each context and even down to each route using MDC logging

  1. Modify the org.ops4j.pax.logging.cfg file adding camel-mdc to the root logger and creating an additional log4j appender to create a separate log file for each camel context context and route.

    org.ops4j.pax.logging.cfg
    # Root logger
    log4j.rootLogger = INFO, out, osgi:VmLogAppender, camel-mdc
    
    ...
     
    # Camel MDC appender for creating separate log files for each camel context
    log4j.appender.camel-mdc=org.apache.log4j.sift.MDCSiftingAppender
    log4j.appender.camel-mdc.key=camel.contextId
    log4j.appender.camel-mdc.default=unknown
    log4j.appender.camel-mdc.appender=org.apache.log4j.FileAppender
    log4j.appender.camel-mdc.appender.layout=org.apache.log4j.PatternLayout
    log4j.appender.camel-mdc.appender.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n
    log4j.appender.camel-mdc.appender.file=${karaf.data}/log/$\\{camel.contextId\\}.log
    log4j.appender.camel-mdc.appender.append=true
    
    ...
    OR for separate route logs use the following
    ...
    # Camel MDC appender for creating separate log files for each camel route into directories based on the camel context
    log4j.appender.camel-mdc=org.apache.log4j.sift.MDCSiftingAppender
    log4j.appender.camel-mdc.key=camel.routeId
    log4j.appender.camel-mdc.key=camel.contextId
    log4j.appender.camel-mdc.default=unknown
    log4j.appender.camel-mdc.appender=org.apache.log4j.FileAppender
    log4j.appender.camel-mdc.appender.layout=org.apache.log4j.PatternLayout
    log4j.appender.camel-mdc.appender.layout.ConversionPattern=%d{ISO8601} | %-5.5p | %-16.16t | %-32.32c{1} | ContextID: %X{camel.contextId} | RouteId: %X{camel.routeId} | %m%n
    log4j.appender.camel-mdc.appender.file=${karaf.data}/log/$\\{camel.contextId\\}/$\\{camel.routeId\\}.log
    log4j.appender.camel-mdc.appender.append=true
    
    

 

http://camel.apache.org/mdc-logging.html

https://karaf.apache.org/manual/latest/users-guide/log.html

http://blog.nanthrax.net/2014/08/mdc-logging-with-apache-karaf-and-camel/