{"id":363,"date":"2025-10-16T08:46:07","date_gmt":"2025-10-16T08:46:07","guid":{"rendered":"https:\/\/logsmith.io\/?p=363"},"modified":"2025-10-16T08:46:08","modified_gmt":"2025-10-16T08:46:08","slug":"when-cim-mapping-goes-sideways","status":"publish","type":"post","link":"https:\/\/logsmith.io\/index.php\/2025\/10\/16\/when-cim-mapping-goes-sideways\/","title":{"rendered":"When CIM Mapping Goes Sideways: Lessons from a Broken Detection"},"content":{"rendered":"\n<h3 class=\"wp-block-heading\">\ud83d\udd0e Introduction<\/h3>\n\n\n\n<p>Everything looked good on paper:<br>\u2705 The detection rule was written.<br>\u2705 The sourcetype was CIM-mapped.<br>\u2705 The data model was enabled.<\/p>\n\n\n\n<p>And still\u2026 nothing.<\/p>\n\n\n\n<p>No alerts. No results. No clue what was broken.<\/p>\n\n\n\n<p>This is a breakdown of a real-world situation where CIM mapping <strong>looked<\/strong> correct but failed in practice \u2014 and what it taught me about the <strong>gap between mapped and usable<\/strong>.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83e\uddf5 The Scenario<\/h3>\n\n\n\n<p>This was part of a Splunk ES deployment where we expected detections to fire from the <code>Web<\/code> data model. We were using correlation searches based on known attack behaviours \u2014 but they never triggered, even when tested with known-good log examples.<\/p>\n\n\n\n<p>That\u2019s where I had to <strong>go lower level<\/strong> and look past the green lights in the CIM Mapping Editor.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\udee0 The Investigation<\/h3>\n\n\n\n<p>I started by <strong>manually searching the data<\/strong> using traditional <code>index=<\/code> queries, like:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>index=web sourcetype=your_source\n| table _time host sourcetype action http_method status\n<\/code><\/pre>\n\n\n\n<p>This gave a raw, unfiltered view of what was really going on with the events. And here\u2019s what I found:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>\u2705 The sourcetype was mapped in the CIM mapping interface<\/li>\n\n\n\n<li>\u274c But the <code>action<\/code> field was always populated as <code>\"unknown\"<\/code><\/li>\n\n\n\n<li>\u274c The <code>http_method<\/code> field didn\u2019t exist at all<\/li>\n\n\n\n<li>\u274c <code>status<\/code> was there but with inconsistent formatting (sometimes numeric, sometimes text)<\/li>\n<\/ul>\n\n\n\n<p>So while CIM was <strong>technically applied<\/strong>, it wasn\u2019t <strong>functionally useful<\/strong>.<br>The detection logic was relying on specific fields that either didn\u2019t exist or weren\u2019t populated correctly.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\u26a1 Why This Happens<\/h3>\n\n\n\n<p>Splunk CIM relies on <strong>field names<\/strong> being present and correctly populated \u2014 but it does <strong>not validate field quality or value fidelity<\/strong>. That\u2019s on you.<\/p>\n\n\n\n<p>CIM Mapping \u2260 Data Quality<br>Field Exists \u2260 Field Is Usable<br>Green Tick \u2260 Green Light<\/p>\n\n\n\n<p>In our case, the data source had been onboarded with minimal effort. It was mapped to the data model just enough to satisfy the interface, but <strong>no one checked<\/strong> whether the fields had the right data.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\u2705 The Fix<\/h3>\n\n\n\n<p>Once I confirmed the problem, I took these steps:<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">1. <strong>Direct inspection with index searches<\/strong><\/h4>\n\n\n\n<p>To truly understand field presence, formatting, and value distribution \u2014 something <code>tstats<\/code> can\u2019t show \u2014 I used raw index queries to explore the data:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>index=web sourcetype=your_source\n| stats count by http_method, action\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">2. <strong>Fixing field names and values<\/strong><\/h4>\n\n\n\n<p>Using <strong>field aliases<\/strong>, <strong>calculated fields<\/strong>, and <strong>transforms<\/strong>, I re-mapped useful data into the right CIM field names.<\/p>\n\n\n\n<p>Example:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>Calculated Field: http_method = coalesce(method, request_method)\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">3. <strong>Validation searches<\/strong><\/h4>\n\n\n\n<p>I created searches that show the frequency and variance of expected fields.<br>This helps detect when a data source degrades or becomes incomplete over time:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>index=web sourcetype=your_source\n| stats dc(http_method) AS methods dc(status) AS statuses\n<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">4. <strong>Dashboards for field-level coverage<\/strong><\/h4>\n\n\n\n<p>To help the SOC and platform team monitor field presence across logs \u2014 not just data volumes.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83c\udfaf Key Takeaways<\/h3>\n\n\n\n<h4 class=\"wp-block-heading\">\u2705 Don\u2019t assume CIM is \u201cset and forget\u201d<\/h4>\n\n\n\n<p>It\u2019s not enough to see your sourcetype listed in the CIM Mapping interface. <strong>You must inspect the actual events.<\/strong><\/p>\n\n\n\n<h4 class=\"wp-block-heading\">\u2705 Validate the quality of mapped fields<\/h4>\n\n\n\n<p>Check if fields are present <strong>and<\/strong> contain meaningful, consistent data. Empty or default fields (like <code>\"unknown\"<\/code>) can break your detections silently.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">\u2705 Build validation as a habit<\/h4>\n\n\n\n<p>Field validation should be part of <strong>your onboarding checklist<\/strong>.<br>It&#8217;s a low-effort, high-impact way to catch data issues before they reach production.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\">\ud83d\udd1a Final Thought<\/h3>\n\n\n\n<p>Splunk ES is only as good as the data it relies on.<br>CIM compliance is a <strong>starting point<\/strong>, not a guarantee.<\/p>\n\n\n\n<p>So before you enable that next detection rule, ask yourself:<br><strong>Can the data really support it?<\/strong><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\ud83d\udd0e Introduction Everything looked good on paper:\u2705 The detection rule was written.\u2705 The sourcetype was CIM-mapped.\u2705 The data model was&#8230; <\/p>\n<div class=\"art-el-more\"><a href=\"https:\/\/logsmith.io\/index.php\/2025\/10\/16\/when-cim-mapping-goes-sideways\/\" class=\"art-link art-color-link art-w-chevron\">Read more<\/a><\/div>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"iawp_total_views":18,"footnotes":""},"categories":[28],"tags":[32,30,33],"class_list":["post-363","post","type-post","status-publish","format-standard","hentry","category-case-study","tag-case-study","tag-detection-engineering","tag-log-normalisation"],"acf":[],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/posts\/363","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/comments?post=363"}],"version-history":[{"count":2,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/posts\/363\/revisions"}],"predecessor-version":[{"id":365,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/posts\/363\/revisions\/365"}],"wp:attachment":[{"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/media?parent=363"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/categories?post=363"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/logsmith.io\/index.php\/wp-json\/wp\/v2\/tags?post=363"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}