feat: improve UX with neutral messaging and fail_on_diff option

- Change '⚠️ API Differences Detected' to '📋 API Changes Detected' for less alarming tone
- Remove <details> nesting for diffs - show them directly for easier reading
- Add fail_on_diff option to fail the action when API changes are detected (for release validation)
- Show ref names (branch/tag) instead of just SHA in reports where possible
- Add links to language starter repos and github-mcp-server in README
- Update README overview to focus on visibility rather than alarming about breaking changes

Author: SamMorrowDrums

README.md

@@ -4,15 +4,15 @@
 [![GitHub release](https://img.shields.io/github/v/release/SamMorrowDrums/mcp-conformance-action)](https://github.com/SamMorrowDrums/mcp-conformance-action/releases)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
-A GitHub Action for detecting changes to [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server **public interfaces**. This action compares the current branch against a baseline to surface any changes to your server's exposed tools, resources, prompts, and capabilities—helping you catch unintended breaking changes and document intentional API evolution.
+A GitHub Action for detecting changes to [Model Context Protocol (MCP)](https://modelcontextprotocol.io/) server **public interfaces**. This action compares the current branch against a baseline to surface any changes to your server's exposed tools, resources, prompts, and capabilities—helping you document API evolution and catch unintended modifications.
 
 ## Overview
 
-MCP servers expose a **public interface** to AI assistants: tools (with their input schemas), resources, prompts, and server capabilities. As your server evolves, changes to this interface can break clients or alter expected behavior. This action automates public interface comparison by:
+MCP servers expose a **public interface** to AI assistants: tools (with their input schemas), resources, prompts, and server capabilities. As your server evolves, changes to this interface are worth tracking. This action automates public interface comparison by:
 
 1. Building your MCP server from both the current branch and a baseline (merge-base, tag, or specified ref)
 2. Querying both versions for their complete public interface (tools, resources, prompts, capabilities)
-3. Generating a detailed diff report showing exactly what changed
+3. Generating a diff report showing exactly what changed
 4. Surfacing results directly in GitHub's Job Summary
 
 This is **not** about testing internal logic or correctness—it's about visibility into what your server _advertises_ to clients.
@@ -200,6 +200,8 @@ Either `start_command` (for stdio) or `server_url` (for HTTP) must be provided,
 | Input | Description | Default |
 |-------|-------------|---------|
 | `compare_ref` | Git ref to compare against. Auto-detects merge-base on PRs or previous tag on tag pushes if not specified. | `""` |
+| `fail_on_diff` | Fail the action if API changes are detected. Useful for release validation workflows. | `false` |
+| `fail_on_error` | Fail the action if probe errors occur (connection failures, etc.) | `true` |
 
 ### Configuration Object Schema
 
@@ -354,6 +356,21 @@ Specify any git ref to compare against:
     compare_ref: v1.0.0
 ```
 
+### Failing on Changes (Release Validation)
+
+For release workflows where you want to ensure no API changes, use `fail_on_diff`:
+
+```yaml
+- uses: SamMorrowDrums/mcp-conformance-action@v2
+  with:
+    setup_node: true
+    install_command: npm ci
+    build_command: npm run build
+    start_command: node dist/stdio.js
+    compare_ref: v1.0.0
+    fail_on_diff: true  # Action fails if any API changes are detected
+```
+
 ## Artifacts and Reports
 
 The action produces:
@@ -439,3 +456,17 @@ Contributions are welcome. Please read [CONTRIBUTING.md](CONTRIBUTING.md) for gu
 - [MCP TypeScript SDK](https://github.com/modelcontextprotocol/typescript-sdk)
 - [MCP Python SDK](https://github.com/modelcontextprotocol/python-sdk)
 - [MCP Go SDK](https://github.com/modelcontextprotocol/go-sdk)
+
+### Example Configurations
+
+Working examples of this action in various languages:
+
+| Language | Repository | Workflow |
+|----------|------------|----------|
+| TypeScript | [mcp-typescript-starter](https://github.com/SamMorrowDrums/mcp-typescript-starter) | [conformance.yml](https://github.com/SamMorrowDrums/mcp-typescript-starter/blob/main/.github/workflows/conformance.yml) |
+| Python | [mcp-python-starter](https://github.com/SamMorrowDrums/mcp-python-starter) | [conformance.yml](https://github.com/SamMorrowDrums/mcp-python-starter/blob/main/.github/workflows/conformance.yml) |
+| Go | [mcp-go-starter](https://github.com/SamMorrowDrums/mcp-go-starter) | [conformance.yml](https://github.com/SamMorrowDrums/mcp-go-starter/blob/main/.github/workflows/conformance.yml) |
+| Rust | [mcp-rust-starter](https://github.com/SamMorrowDrums/mcp-rust-starter) | [conformance.yml](https://github.com/SamMorrowDrums/mcp-rust-starter/blob/main/.github/workflows/conformance.yml) |
+| C# | [mcp-csharp-starter](https://github.com/SamMorrowDrums/mcp-csharp-starter) | [conformance.yml](https://github.com/SamMorrowDrums/mcp-csharp-starter/blob/main/.github/workflows/conformance.yml) |
+
+For a production example, see [github-mcp-server](https://github.com/github/github-mcp-server).

action.yml

@@ -104,6 +104,10 @@ inputs:
     description: 'Fail the action if probe errors occur (not just API differences)'
     required: false
     default: 'true'
+  fail_on_diff:
+    description: 'Fail the action if API differences are detected (useful for release validation)'
+    required: false
+    default: 'false'
   env_vars:
     description: 'Environment variables (newline-separated KEY=VALUE pairs)'
     required: false
@@ -232,15 +236,9 @@ runs:
         echo "" >> $GITHUB_STEP_SUMMARY
         
         if [ "${{ steps.conformance.outputs.status }}" = "passed" ]; then
-          echo "**All conformance tests passed** - No behavioral differences detected." >> $GITHUB_STEP_SUMMARY
+          echo "**No API changes detected** between the branches." >> $GITHUB_STEP_SUMMARY
         else
-          echo "**Differences detected** - Review the diffs above to ensure changes are intentional." >> $GITHUB_STEP_SUMMARY
-          echo "" >> $GITHUB_STEP_SUMMARY
-          echo "Common expected differences:" >> $GITHUB_STEP_SUMMARY
-          echo "- New tools added" >> $GITHUB_STEP_SUMMARY
-          echo "- Tool descriptions updated" >> $GITHUB_STEP_SUMMARY
-          echo "- New resources or prompts" >> $GITHUB_STEP_SUMMARY
-          echo "- Capability changes" >> $GITHUB_STEP_SUMMARY
+          echo "**API changes detected** — review above to confirm they are expected." >> $GITHUB_STEP_SUMMARY
         fi
 
     - name: Upload conformance report

dist/git.d.ts

@@ -30,3 +30,8 @@ export declare function checkout(ref: string): Promise<void>;
  * Checkout previous branch/ref
  */
 export declare function checkoutPrevious(): Promise<void>;
+/**
+ * Get a display-friendly name for a ref.
+ * Returns branch/tag name if available, otherwise the short SHA.
+ */
+export declare function getRefDisplayName(ref: string): Promise<string>;

dist/index.js

@@ -35472,6 +35472,76 @@ async function checkoutPrevious() {
         // Ignore errors
     }
 }
+/**
+ * Get a display-friendly name for a ref.
+ * Returns branch/tag name if available, otherwise the short SHA.
+ */
+async function getRefDisplayName(ref) {
+    // If it's already a readable name (not a SHA), return it
+    if (!ref.match(/^[a-f0-9]{40}$/i) && !ref.match(/^[a-f0-9]{7,}$/i)) {
+        // It's likely already a branch/tag name
+        return ref;
+    }
+    // Try to find a branch name pointing to this ref
+    let output = "";
+    try {
+        await exec.exec("git", ["branch", "--points-at", ref, "--format=%(refname:short)"], {
+            silent: true,
+            listeners: {
+                stdout: (data) => {
+                    output += data.toString();
+                },
+            },
+        });
+        const branches = output.trim().split("\n").filter(Boolean);
+        if (branches.length > 0) {
+            // Prefer main/master if available
+            if (branches.includes("main"))
+                return "main";
+            if (branches.includes("master"))
+                return "master";
+            return branches[0];
+        }
+    }
+    catch {
+        // Ignore errors
+    }
+    // Try to find a tag pointing to this ref
+    output = "";
+    try {
+        await exec.exec("git", ["tag", "--points-at", ref], {
+            silent: true,
+            listeners: {
+                stdout: (data) => {
+                    output += data.toString();
+                },
+            },
+        });
+        const tags = output.trim().split("\n").filter(Boolean);
+        if (tags.length > 0) {
+            return tags[0];
+        }
+    }
+    catch {
+        // Ignore errors
+    }
+    // Fall back to short SHA
+    output = "";
+    try {
+        await exec.exec("git", ["rev-parse", "--short", ref], {
+            silent: true,
+            listeners: {
+                stdout: (data) => {
+                    output += data.toString();
+                },
+            },
+        });
+        return output.trim() || ref;
+    }
+    catch {
+        return ref.substring(0, 7);
+    }
+}
 
 // EXTERNAL MODULE: external "path"
 var external_path_ = __nccwpck_require__(6928);
@@ -53804,10 +53874,10 @@ async function runAllTests(ctx) {
         result.diffs = compareResults(branchFiles, baseFiles);
         result.hasDifferences = result.diffs.size > 0;
         if (result.hasDifferences) {
-            lib_core.warning(`⚠️ Configuration ${config.name}: ${result.diffs.size} differences found`);
+            lib_core.info(`📋 Configuration ${config.name}: ${result.diffs.size} change(s) found`);
         }
         else {
-            lib_core.info(`✅ Configuration ${config.name}: no differences`);
+            lib_core.info(`✅ Configuration ${config.name}: no changes`);
         }
         // Save individual result
         const resultPath = external_path_.join(ctx.workDir, ".conformance-results", `${config.name}.json`);
@@ -53871,14 +53941,14 @@ function generateMarkdownReport(report) {
     lines.push("");
     // Overall status
     if (report.diffCount === 0) {
-        lines.push("## ✅ All Conformance Tests Passed");
+        lines.push("## ✅ No API Changes");
         lines.push("");
-        lines.push("No API differences detected between the current branch and the comparison ref.");
+        lines.push("No differences detected between the current branch and the comparison ref.");
     }
     else {
-        lines.push("## ⚠️ API Differences Detected");
+        lines.push("## 📋 API Changes Detected");
         lines.push("");
-        lines.push(`${report.diffCount} configuration(s) have API differences that may indicate breaking changes.`);
+        lines.push(`${report.diffCount} configuration(s) have changes. Review below to ensure they are intentional.`);
     }
     lines.push("");
     // Per-configuration results
@@ -53893,18 +53963,15 @@ function generateMarkdownReport(report) {
         lines.push(`- **Base Time:** ${formatTime(result.baseTime)}`);
         lines.push("");
         if (result.hasDifferences) {
-            lines.push("#### Differences");
+            lines.push("#### Changes");
             lines.push("");
             for (const [endpoint, diff] of result.diffs) {
-                lines.push(`<details>`);
-                lines.push(`<summary><strong>${endpoint}</strong></summary>`);
+                lines.push(`**${endpoint}**`);
                 lines.push("");
                 lines.push("```diff");
                 lines.push(diff);
                 lines.push("```");
                 lines.push("");
-                lines.push("</details>");
-                lines.push("");
             }
         }
         else {
@@ -53969,14 +54036,14 @@ function saveReport(report, markdown, outputDir) {
 function generatePRSummary(report) {
     const lines = [];
     if (report.diffCount === 0) {
-        lines.push("## ✅ MCP Conformance: All Tests Passed");
+        lines.push("## ✅ MCP Conformance: No Changes");
         lines.push("");
-        lines.push(`Tested ${report.results.length} configuration(s) - no API breaking changes detected.`);
+        lines.push(`Tested ${report.results.length} configuration(s) - no API changes detected.`);
     }
     else {
-        lines.push("## ⚠️ MCP Conformance: API Differences Detected");
+        lines.push("## 📋 MCP Conformance: API Changes Detected");
         lines.push("");
-        lines.push(`**${report.diffCount}** of ${report.results.length} configuration(s) have differences.`);
+        lines.push(`**${report.diffCount}** of ${report.results.length} configuration(s) have changes.`);
         lines.push("");
         lines.push("### Changed Endpoints");
         lines.push("");
@@ -54054,6 +54121,7 @@ function getInputs() {
         // Test configuration
         compareRef: getInput("compare_ref"),
         failOnError: getBooleanInput("fail_on_error") !== false, // default true
+        failOnDiff: getBooleanInput("fail_on_diff") === true, // default false
         envVars: getInput("env_vars"),
         serverTimeout: parseInt(getInput("server_timeout") || "30000", 10),
     };
@@ -54188,10 +54256,11 @@ async function run() {
         // Determine comparison ref
         const currentBranch = await getCurrentBranch();
         const compareRef = await determineCompareRef(inputs.compareRef, process.env.GITHUB_REF);
+        const compareRefDisplay = await getRefDisplayName(compareRef);
         lib_core.info("");
         lib_core.info(`📊 Comparison:`);
         lib_core.info(`  Current: ${currentBranch}`);
-        lib_core.info(`  Compare: ${compareRef}`);
+        lib_core.info(`  Compare: ${compareRefDisplay}${compareRefDisplay !== compareRef ? ` (${compareRef.substring(0, 7)})` : ""}`);
         // Run all tests
         lib_core.info("");
         lib_core.info("🧪 Running conformance tests...");
@@ -54204,7 +54273,7 @@ async function run() {
         // Generate and save report
         lib_core.info("");
         lib_core.info("📝 Generating report...");
-        const report = generateReport(results, currentBranch, compareRef);
+        const report = generateReport(results, currentBranch, compareRefDisplay);
         const markdown = generateMarkdownReport(report);
         saveReport(report, markdown, workDir);
         // Set final status
@@ -54216,13 +54285,18 @@ async function run() {
             lib_core.setFailed(`❌ Probe errors occurred in: ${errorConfigs.join(", ")}`);
         }
         else if (report.diffCount > 0) {
-            lib_core.warning(`⚠️ ${report.diffCount} configuration(s) have API differences`);
+            if (inputs.failOnDiff) {
+                lib_core.setFailed(`❌ ${report.diffCount} configuration(s) have API changes`);
+            }
+            else {
+                lib_core.info(`📋 ${report.diffCount} configuration(s) have API changes`);
+            }
             if (hasErrors) {
                 lib_core.warning("Some configurations had probe errors (fail_on_error is disabled)");
             }
         }
         else {
-            lib_core.info("✅ All conformance tests passed!");
+            lib_core.info("✅ All conformance tests passed - no API changes detected");
         }
     }
     catch (error) {

dist/types.d.ts

@@ -45,6 +45,7 @@ export interface ActionInputs {
     customMessages: CustomMessage[];
     compareRef: string;
     failOnError: boolean;
+    failOnDiff: boolean;
     envVars: string;
     serverTimeout: number;
     httpStartCommand: string;

src/git.ts

@@ -191,3 +191,72 @@ export async function checkoutPrevious(): Promise<void> {
     // Ignore errors
   }
 }
+
+/**
+ * Get a display-friendly name for a ref.
+ * Returns branch/tag name if available, otherwise the short SHA.
+ */
+export async function getRefDisplayName(ref: string): Promise<string> {
+  // If it's already a readable name (not a SHA), return it
+  if (!ref.match(/^[a-f0-9]{40}$/i) && !ref.match(/^[a-f0-9]{7,}$/i)) {
+    // It's likely already a branch/tag name
+    return ref;
+  }
+
+  // Try to find a branch name pointing to this ref
+  let output = "";
+  try {
+    await exec.exec("git", ["branch", "--points-at", ref, "--format=%(refname:short)"], {
+      silent: true,
+      listeners: {
+        stdout: (data) => {
+          output += data.toString();
+        },
+      },
+    });
+    const branches = output.trim().split("\n").filter(Boolean);
+    if (branches.length > 0) {
+      // Prefer main/master if available
+      if (branches.includes("main")) return "main";
+      if (branches.includes("master")) return "master";
+      return branches[0];
+    }
+  } catch {
+    // Ignore errors
+  }
+
+  // Try to find a tag pointing to this ref
+  output = "";
+  try {
+    await exec.exec("git", ["tag", "--points-at", ref], {
+      silent: true,
+      listeners: {
+        stdout: (data) => {
+          output += data.toString();
+        },
+      },
+    });
+    const tags = output.trim().split("\n").filter(Boolean);
+    if (tags.length > 0) {
+      return tags[0];
+    }
+  } catch {
+    // Ignore errors
+  }
+
+  // Fall back to short SHA
+  output = "";
+  try {
+    await exec.exec("git", ["rev-parse", "--short", ref], {
+      silent: true,
+      listeners: {
+        stdout: (data) => {
+          output += data.toString();
+        },
+      },
+    });
+    return output.trim() || ref;
+  } catch {
+    return ref.substring(0, 7);
+  }
+}