Support attribute paths with dots in custom policies (YAML custom policies)

[mbukh]

Currently, the attribute field in yaml custom policies does not support paths that include dots ("."). This makes it difficult to reference nested keys that have dots in their names, such as:

valuesObject:
  ingress:
    annotations:
      external-dns.alpha.kubernetes.io/ingress-hostname-source: defined-hosts-only

To reference this path I would do:

definition:
  attribute: valuesObject.ingress.annotations.external-dns.alpha.kubernetes.io/ingress-hostname-source

Which will definitely fail.

We should add support for either:

1. Escaping dots with a backslash like attribute\.with\.dots
2. Using square bracket notation like ['attribute.with.dots']
3. Both 1) and 2)

For example:

definition:
  # attribute: valuesObject.ingress.annotations.external-dns\\.alpha\\.kubernetes\\.io/ingress-hostname-source
  attribute: |-
    valuesObject.ingress.annotations.external-dns\.alpha\.kubernetes\.io/ingress-hostname-source

or

definition:
  attribute: valuesObject.ingress.annotations.external-dns['alpha.kubernetes.io/ingress-hostname-source']

This will allow more flexibility in referencing nested keys in custom policies.

[mbukh]

Solution proposal for escaped dot "\." in checkov/common/checks_infra/solvers/attribute_solvers/base_attribute_solver.py:

def get_attribute_patterns(attribute: str) -> Tuple[Pattern[str], Pattern[str]]:
    index_pattern = r"[\d]+"
    split_by_dots = re.split(r'(?<!\\)\.', attribute)

    pattern_parts = []
    pattern_parts_without_index = []
    for attr_part in split_by_dots:
        if attr_part == "*":
            pattern_parts.append(index_pattern)
        else:
            attr_part = re.escape(attr_part.replace('\\.', '.'))
            attr_part_pattern = f"({attr_part})"
            pattern_parts.append(attr_part_pattern)
            pattern_parts_without_index.append(attr_part_pattern)

    pattern = f'^{"[.]".join(pattern_parts)}$'
    pattern_with_index = re.compile(pattern)

    pattern = f'^{"[.]".join(pattern_parts_without_index)}$'  
    pattern_without_index = re.compile(pattern)

    return pattern_with_index, pattern_without_index

1. re.split(r'(?<!\\)\.', attribute) will split on dots, but not on escaped dots preceded by a backslash.
2. re.escape(attr_part.replace('\\.', '.')) replaces escaped dots \. with a literal dot . and then escapes it for the regex.

This allows attribute paths like valuesObject.ingress.annotations.external-dns.alpha.kubernetes.io/ingress-hostname-source to be properly handled without splitting on the escaped dots.

[mbukh]

Solution proposal for brackets with quotes (single or double) in checkov/common/checks_infra/solvers/attribute_solvers/base_attribute_solver.py:

def get_attribute_patterns(attribute: str) -> Tuple[Pattern[str], Pattern[str]]:
    index_pattern = r"[\d]+"
    bracket_pattern = r'\[[\'"].*?[\'"]\]'
    
    parts = re.split(f'({bracket_pattern})', attribute)
    split_by_dots = []
    for part in parts:
        if re.match(bracket_pattern, part):
            split_by_dots.append(part)
        else:
            split_by_dots.extend(part.split('.'))

    pattern_parts = []
    pattern_parts_without_index = []
    for attr_part in split_by_dots:
        if attr_part == "*":
            pattern_parts.append(index_pattern)
        else:
            if re.match(bracket_pattern, attr_part):
                attr_part = attr_part[1:-1]  # remove brackets
                attr_part = re.escape(attr_part[1:-1])  # remove quotes and escape
            attr_part_pattern = f"({attr_part})"
            pattern_parts.append(attr_part_pattern)
            pattern_parts_without_index.append(attr_part_pattern)

    pattern = f'^{"[.]".join(pattern_parts)}$'
    pattern_with_index = re.compile(pattern)

    pattern = f'^{"[.]".join(pattern_parts_without_index)}$'
    pattern_without_index = re.compile(pattern)

    return pattern_with_index, pattern_without_index

1. re.split(f'({bracket_pattern})', attribute) separates the bracketed parts from the rest.

This allows attribute paths with square bracket notation like valuesObject.ingress.annotations['external-dns.alpha.kubernetes.io/ingress-hostname-source'] or valuesObject.ingress["annotations.with.dots"].

[lirshindalman]

Hi @mbukh thank you for reaching out and the detailed explanation. it's a valid point. you are welcome to open PR with the suggestion fix and we will review it.