Comprehensive Test Cases for Edge Cases and Invalid Manifests

Objective

Expand the test suite with comprehensive coverage of edge cases, invalid manifests, and real-world failure scenarios that specification consumers will encounter.

Scope

Add extensive test cases covering:

• Malformed YAML/JSON parsing
• Missing required fields at various nesting levels
• Invalid data types and format violations
• Boundary conditions (empty arrays, max lengths, etc.)
• Cross-field validation (conditional requirements)
• Security vulnerabilities in manifest content

Technical Approach

// tests/validation-suite/edge-cases.test.ts
describe('Edge Cases and Invalid Manifests', () => {
  describe('Malformed YAML', () => {
    test('Invalid YAML syntax should fail gracefully', () => {
      const invalidYaml = 'ossaVersion: 1.0\n  agent:\n    id: test'; // Bad indentation
      expect(() => yaml.parse(invalidYaml)).toThrow();
    });

    test('Mixed tabs and spaces should be rejected', () => {
      // Real-world issue: developers mixing tabs/spaces
    });
  });

  describe('Missing Required Fields', () => {
    test('Missing top-level ossaVersion', () => {
      const manifest = { agent: { /* complete agent */ } };
      expect(validate(manifest)).toBe(false);
      expect(validate.errors[0].message).toContain('ossaVersion');
    });

    test('Missing agent.id should fail', () => {
      const manifest = createMinimalAgent({ id: undefined });
      expect(validate(manifest)).toBe(false);
    });

    test('Missing nested runtime.type', () => {
      const manifest = createMinimalAgent({ 
        runtime: { image: 'test:1.0' } // type missing
      });
      expect(validate(manifest)).toBe(false);
    });
  });

  describe('Boundary Conditions', () => {
    test('Empty capabilities array should fail', () => {
      // minItems: 1 enforcement
    });

    test('Agent ID at maximum length (253 characters)', () => {
      const longId = 'a'.repeat(253); // DNS-1123 subdomain max
      const manifest = createMinimalAgent({ id: longId });
      expect(validate(manifest)).toBe(true);
    });

    test('Agent ID exceeding maximum length should fail', () => {
      const tooLong = 'a'.repeat(254);
      const manifest = createMinimalAgent({ id: tooLong });
      expect(validate(manifest)).toBe(false);
    });

    test('Empty string values should fail', () => {
      const manifest = createMinimalAgent({ name: '' });
      expect(validate(manifest)).toBe(false);
    });
  });

  describe('Type Validation', () => {
    test('Numeric version instead of string should fail', () => {
      const manifest = createMinimalAgent({ version: 1.0 as any });
      expect(validate(manifest)).toBe(false);
    });

    test('Array instead of object should fail', () => {
      const manifest = createMinimalAgent({ runtime: [] as any });
      expect(validate(manifest)).toBe(false);
    });
  });

  describe('Format Validation', () => {
    test('Invalid semver formats', () => {
      const invalidVersions = ['v1.0.0', '1.0', 'latest', '1.x.x'];
      invalidVersions.forEach(version => {
        const manifest = createMinimalAgent({ version });
        expect(validate(manifest)).toBe(false);
      });
    });

    test('Invalid DNS-1123 subdomain for agent ID', () => {
      const invalidIds = [
        'UPPERCASE',           // uppercase not allowed
        'has_underscore',      // underscores not allowed
        'ends-with-dash-',     // can't end with dash
        '-starts-with-dash',   // can't start with dash
        'has..consecutive',    // consecutive dots
        'has spaces'           // spaces not allowed
      ];
      invalidIds.forEach(id => {
        const manifest = createMinimalAgent({ id });
        expect(validate(manifest)).toBe(false);
      });
    });
  });

  describe('Security Validation', () => {
    test('Injection attempts in string fields', () => {
      const xssAttempt = '<script>alert(1)</script>';
      const manifest = createMinimalAgent({ name: xssAttempt });
      // Should validate structurally (consumers handle sanitization)
      expect(validate(manifest)).toBe(true);
    });

    test('Extremely deep nesting should not cause stack overflow', () => {
      const deeplyNested = createDeeplyNestedObject(1000);
      expect(() => validate(deeplyNested)).not.toThrow();
    });

    test('Extremely large manifest should fail gracefully', () => {
      const largeManifest = createManifestWithManyCapabilities(10000);
      expect(validate(largeManifest)).toBe(false); // Should have size limits
    });
  });

  describe('Cross-Field Validation', () => {
    test('K8s runtime requires image field', () => {
      const manifest = createMinimalAgent({
        runtime: { type: 'k8s' } // missing image
      });
      expect(validate(manifest)).toBe(false);
    });

    test('Compliance frameworks require encryption', () => {
      const manifest = createMinimalAgent({
        policies: {
          compliance: ['fedramp-moderate'],
          encryption: false // Should fail
        }
      });
      // This requires conditional schema support
    });
  });

  describe('Real-World Failure Scenarios', () => {
    test('Copy-paste errors with wrong indentation', () => {
      const badYaml = `
ossaVersion: 1.0
agent:
id: test-agent  # Missing indentation
      `;
      expect(() => yaml.parse(badYaml)).toThrow();
    });

    test('Mixed YAML and JSON (common mistake)', () => {
      const mixed = `
ossaVersion: "1.0"
agent: {"id": "test-agent"}
      `;
      // Should parse but validate structure
    });
  });
});

Acceptance Criteria

• 50+ additional test cases covering edge cases
• All boundary conditions tested (min/max lengths, empty arrays, etc.)
• Invalid format testing (semver, DNS-1123, etc.)
• Cross-field validation tests
• Security vulnerability tests (injection, DoS, etc.)
• Real-world failure scenario reproduction
• Test coverage >95% for schema validation
• Comprehensive error message validation
• Performance tests for large manifests

Files to Modify

• tests/validation-suite/edge-cases.test.ts - New comprehensive test file
• tests/validation-suite/security.test.ts - Security-specific tests
• tests/validation-suite/boundary-conditions.test.ts - Boundary testing