Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New AST builder. #135

Merged
merged 33 commits into from
Feb 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
0b3df04
New AST builder.
Washi1337 Nov 5, 2023
3ea949c
Re-enable missing unit tests.
Washi1337 Nov 5, 2023
3b2d013
Flush only when stack is not impure.
Washi1337 Nov 15, 2023
0f6bde0
Fix purity in dummy test arch.
Washi1337 Nov 17, 2023
66d841d
Add stateful block visitor interface.
Washi1337 Nov 19, 2023
9aa1444
Add TreeNodeBase::OnParentChanged.
Washi1337 Nov 19, 2023
6e0180a
Add nested impure expression test.
Washi1337 Nov 19, 2023
fe66cac
Add AstNode::OriginalRange.
Washi1337 Nov 19, 2023
3f6109a
BUGFIX: Check argument range for null before expanding containing range.
Washi1337 Nov 19, 2023
f4c8fe0
Shorten some pattern factory methods.
Washi1337 Dec 25, 2023
4e0ff1e
Strongly typed AST capture groups.
Washi1337 Dec 25, 2023
cf95b06
Add factory methods for ast nodes analogous to the ast patterns.
Washi1337 Dec 25, 2023
eef2ac8
Promote stack intermediate variables to output vars if they are left …
Washi1337 Dec 25, 2023
ab4e6eb
Enable NRT in AST package.
Washi1337 Dec 27, 2023
acb2364
Change default purity of CIL field accesses to false.
Washi1337 Dec 27, 2023
32cc13c
Inline input stack variables if PHI has only one source.
Washi1337 Dec 28, 2023
901c9ff
Lazy init the synthetic variable collections.
Washi1337 Dec 28, 2023
36b8072
Extract AstFormatter and instruction formatters.
Washi1337 Dec 28, 2023
1dc6f87
Update README.md
Washi1337 Dec 28, 2023
7ed767f
Fix AstNodeWalker, add parameterless version of IAstNodeVisitor.
Washi1337 Jan 4, 2024
9f85b59
Reduce allocs in AST building. Add ToAst extension method to Architec…
Washi1337 Jan 4, 2024
09e113f
Update AST read/written variable finders.
Washi1337 Jan 4, 2024
3f13ae0
Reduce casting in ast builder.
Washi1337 Jan 26, 2024
786c926
Add block statement models.
Washi1337 Feb 1, 2024
32fc279
Add compilation unit builder.
Washi1337 Feb 1, 2024
e6c2f8a
Add variable cross-referencing.
Washi1337 Feb 1, 2024
fe30e67
BUGFIX: Register/unregister variables added/remove from existing assi…
Washi1337 Feb 1, 2024
aee24b0
Rename ToAst -> Lift
Washi1337 Feb 1, 2024
4c340ab
BUGFIX: Clone x-ref list to prevent side-effects in caller.
Washi1337 Feb 1, 2024
591698d
Resolve xml warnings.
Washi1337 Feb 1, 2024
4cf10d6
Add original offset ranges to block statements.
Washi1337 Feb 2, 2024
97fdd2f
BUGFIX: Prevent tree nodes from being added to themselves as children.
Washi1337 Feb 2, 2024
e00a467
BUGFIX: Map EH region successors to their unbreakable path view nodes.
Washi1337 Feb 2, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,19 +14,21 @@ Main Features
- Data flow analysis
- Create data flow graphs
- Inspect stack and variable dependencies of instructions.
- AST building
- Lift control flow graphs to Abstract Syntax Trees (ASTs).
- Unified generic API.
- Serialize any kind of graph to the dot file format.
- Adding a new platform for flow analysis requires minimal effort


Supported platforms:

| Architecture | Back-end | Control Flow | Data Flow | Purity Classification | Emulation |
|--------------|---------------------------------------------------------|--------------|-----------|-----------------------|-----------|
| CIL | [AsmResolver](https://github.com/Washi1337/AsmResolver) | ✓ | ✓ | ✓ | ✓ (WIP) |
| CIL | [dnlib](https://github.com/0xd4d/dnlib) | ✓ | ✓ | ✓ | |
| x86 (32-bit) | [Iced](https://github.com/icedland/iced) | ✓ | ✓ | | |
| x86 (64-bit) | [Iced](https://github.com/icedland/iced) | ✓ | ✓ | | |
| Architecture | Back-end | Control Flow | Data Flow | AST | Purity Classification | Emulation |
|--------------|---------------------------------------------------------|--------------|-----------|-----|-----------------------|-----------|
| CIL | [AsmResolver](https://github.com/Washi1337/AsmResolver) | ✓ | ✓ | ✓ | ✓ | ✓ (WIP) |
| CIL | [dnlib](https://github.com/0xd4d/dnlib) | ✓ | ✓ | ✓ | ✓ | |
| x86 (32-bit) | [Iced](https://github.com/icedland/iced) | ✓ | ✓ | ✓ | | |
| x86 (64-bit) | [Iced](https://github.com/icedland/iced) | ✓ | ✓ | ✓ | | |


Compiling
Expand Down Expand Up @@ -54,5 +56,4 @@ See [CONTRIBUTING.md](CONTRIBUTING.md).

Found a bug or have questions?
------------------------------
Please use the [issue tracker](https://github.com/Washi1337/Echo/issues). Try to be as descriptive as possible.

Please use the [issue tracker](https://github.com/Washi1337/Echo/issues). Try to be as descriptive as possible.
34 changes: 34 additions & 0 deletions src/Core/Echo.Ast/Analysis/AstPurityClassifier.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
using Echo.Code;

namespace Echo.Ast.Analysis;

/// <summary>
/// Provides a wrapper around a <see cref="IPurityClassifier{TInstruction}"/> that is able to classify statements
/// and expressions with <typeparamref name="TInstruction"/> instructions by purity.
/// </summary>
/// <typeparam name="TInstruction">The type of instructions the statements store.</typeparam>
public class AstPurityClassifier<TInstruction> : IPurityClassifier<Statement<TInstruction>>
{
/// <summary>
/// Creates a new instance of the <see cref="AstPurityClassifier{TInstruction}"/> class.
/// </summary>
/// <param name="baseClassifier">The base classifier to use for classifying individual instructions in the AST.</param>
public AstPurityClassifier(IPurityClassifier<TInstruction> baseClassifier)
{
BaseClassifier = baseClassifier;
}

/// <summary>
/// Gets the base classifier to use for classifying individual instructions in the AST.
/// </summary>
public IPurityClassifier<TInstruction> BaseClassifier
{
get;
}

/// <inheritdoc />
public Trilean IsPure(in Statement<TInstruction> instruction)
{
return instruction.Accept(AstPurityVisitor<TInstruction>.Instance, BaseClassifier);
}
}
89 changes: 89 additions & 0 deletions src/Core/Echo.Ast/Analysis/AstPurityVisitor.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
using Echo.Code;

namespace Echo.Ast.Analysis;

/// <summary>
/// Provides a mechanism for traversing an AST and determining its purity.
/// </summary>
/// <typeparam name="TInstruction">The type of instructions to store in each expression.</typeparam>
public class AstPurityVisitor<TInstruction> : IAstNodeVisitor<TInstruction, IPurityClassifier<TInstruction>, Trilean>
{
/// <summary>
/// Gets the singleton instance of the <see cref="AstPurityVisitor{TInstruction}"/> class.
/// </summary>
public static AstPurityVisitor<TInstruction> Instance
{
get;
} = new();

/// <inheritdoc />
public Trilean Visit(CompilationUnit<TInstruction> unit, IPurityClassifier<TInstruction> state)
{
return unit.Root.Accept(this, state);
}

/// <inheritdoc />
public Trilean Visit(AssignmentStatement<TInstruction> statement, IPurityClassifier<TInstruction> state) => false;

/// <inheritdoc />
public Trilean Visit(ExpressionStatement<TInstruction> statement, IPurityClassifier<TInstruction> state)
{
return statement.Expression.Accept(this, state);
}

/// <inheritdoc />
public Trilean Visit(PhiStatement<TInstruction> statement, IPurityClassifier<TInstruction> state) => false;

/// <inheritdoc />
public Trilean Visit(BlockStatement<TInstruction> statement, IPurityClassifier<TInstruction> state)
{
var result = Trilean.True;

for (int i = 0; i < statement.Statements.Count && result != Trilean.False; i++)
result &= statement.Statements[i].Accept(this, state);

return result;
}

/// <inheritdoc />
public Trilean Visit(ExceptionHandlerStatement<TInstruction> statement, IPurityClassifier<TInstruction> state)
{
var result = statement.ProtectedBlock.Accept(this, state);

for (int i = 0; i < statement.Handlers.Count && result != Trilean.False; i++)
result &= statement.Handlers[i].Accept(this, state);

return result;
}

/// <inheritdoc />
public Trilean Visit(HandlerClause<TInstruction> clause, IPurityClassifier<TInstruction> state)
{
var result = Trilean.True;

if (clause.Prologue is not null)
result &= clause.Prologue.Accept(this, state);

if (result.ToBooleanOrFalse())
result &= clause.Contents.Accept(this, state);

if (clause.Epilogue is not null && result.ToBooleanOrFalse())
result &= clause.Epilogue.Accept(this, state);

return result;
}

/// <inheritdoc />
public Trilean Visit(InstructionExpression<TInstruction> expression, IPurityClassifier<TInstruction> state)
{
var result = state.IsPure(expression.Instruction);

for (int i = 0; i < expression.Arguments.Count && result != Trilean.False; i++)
result &= expression.Arguments[i].Accept(this, state);

return result;
}

/// <inheritdoc />
public Trilean Visit(VariableExpression<TInstruction> expression, IPurityClassifier<TInstruction> state) => true;
}
41 changes: 31 additions & 10 deletions src/Core/Echo.Ast/Analysis/FlowControlDeterminer.cs
Original file line number Diff line number Diff line change
@@ -1,28 +1,49 @@
using Echo.Code;
using System;
using Echo.Code;

namespace Echo.Ast.Analysis
{
internal sealed class FlowControlDeterminer<TInstruction>
: IAstNodeVisitor<TInstruction, object, InstructionFlowControl>
: IAstNodeVisitor<TInstruction, object?, InstructionFlowControl>
{
private readonly IArchitecture<TInstruction> _isa;

internal FlowControlDeterminer(IArchitecture<TInstruction> isa) =>
_isa = isa;

public InstructionFlowControl Visit(AssignmentStatement<TInstruction> assignmentStatement, object state) =>
assignmentStatement.Expression.Accept(this, state);
public InstructionFlowControl Visit(CompilationUnit<TInstruction> unit, object? state) =>
throw new NotSupportedException();

public InstructionFlowControl Visit(ExpressionStatement<TInstruction> expressionStatement, object state) =>
expressionStatement.Expression.Accept(this, state);
public InstructionFlowControl Visit(AssignmentStatement<TInstruction> statement, object? state) =>
statement.Expression.Accept(this, state);

public InstructionFlowControl Visit(PhiStatement<TInstruction> phiStatement, object state) =>
public InstructionFlowControl Visit(ExpressionStatement<TInstruction> statement, object? state) =>
statement.Expression.Accept(this, state);

public InstructionFlowControl Visit(PhiStatement<TInstruction> statement, object? state) =>
InstructionFlowControl.Fallthrough;

public InstructionFlowControl Visit(InstructionExpression<TInstruction> instructionExpression, object state) =>
_isa.GetFlowControl(instructionExpression.Instruction);
public InstructionFlowControl Visit(BlockStatement<TInstruction> statement, object? state)
{
return statement.Statements.Count > 0
? statement.Statements[statement.Statements.Count - 1].Accept(this, state)
: InstructionFlowControl.Fallthrough;
}

public InstructionFlowControl Visit(ExceptionHandlerStatement<TInstruction> statement, object? state)
{
return statement.ProtectedBlock.Accept(this, state);
}

public InstructionFlowControl Visit(HandlerClause<TInstruction> clause, object? state)
{
throw new NotSupportedException();
}

public InstructionFlowControl Visit(InstructionExpression<TInstruction> expression, object? state) =>
_isa.GetFlowControl(expression.Instruction);

public InstructionFlowControl Visit(VariableExpression<TInstruction> variableExpression, object state) =>
public InstructionFlowControl Visit(VariableExpression<TInstruction> expression, object? state) =>
InstructionFlowControl.Fallthrough;
}
}
41 changes: 41 additions & 0 deletions src/Core/Echo.Ast/Analysis/ReadVariableFinder.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
using System.Buffers;
using System.Collections.Generic;
using Echo.Code;

namespace Echo.Ast.Analysis
{
internal sealed class ReadVariableFinder<TInstruction> : AstNodeListener<TInstruction>
{
private readonly IArchitecture<TInstruction> _architecture;

public ReadVariableFinder(IArchitecture<TInstruction> architecture)
{
_architecture = architecture;
}

internal HashSet<IVariable> Variables { get; } = new();

public override void ExitVariableExpression(VariableExpression<TInstruction> expression)
{
base.ExitVariableExpression(expression);
Variables.Add(expression.Variable);
}

public override void ExitInstructionExpression(InstructionExpression<TInstruction> expression)
{
int count = _architecture.GetReadVariablesCount(expression.Instruction);
if (count == 0)
return;

var variables = ArrayPool<IVariable>.Shared.Rent(count);

int actualCount = _architecture.GetReadVariables(expression.Instruction, variables);
for (int i = 0; i < actualCount; i++)
Variables.Add(variables[i]);

ArrayPool<IVariable>.Shared.Return(variables);

base.ExitInstructionExpression(expression);
}
}
}
18 changes: 0 additions & 18 deletions src/Core/Echo.Ast/Analysis/ReadVariableFinderWalker.cs

This file was deleted.

48 changes: 48 additions & 0 deletions src/Core/Echo.Ast/Analysis/WrittenVariableFinder.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
using System.Buffers;
using System.Collections.Generic;
using Echo.Code;

namespace Echo.Ast.Analysis
{
internal sealed class WrittenVariableFinder<TInstruction> : AstNodeListener<TInstruction>
{
private readonly IArchitecture<TInstruction> _architecture;

public WrittenVariableFinder(IArchitecture<TInstruction> architecture)
{
_architecture = architecture;
}

internal HashSet<IVariable> Variables { get; } = new();

public override void ExitAssignmentStatement(AssignmentStatement<TInstruction> statement)
{
base.ExitAssignmentStatement(statement);
for (int i = 0; i < statement.Variables.Count; i++)
Variables.Add(statement.Variables[i]);
}

public override void ExitPhiStatement(PhiStatement<TInstruction> phiStatement)
{
base.ExitPhiStatement(phiStatement);
Variables.Add(phiStatement.Representative);
}

public override void ExitInstructionExpression(InstructionExpression<TInstruction> expression)
{
int count = _architecture.GetWrittenVariablesCount(expression.Instruction);
if (count == 0)
return;

var variables = ArrayPool<IVariable>.Shared.Rent(count);

int actualCount = _architecture.GetWrittenVariables(expression.Instruction, variables);
for (int i = 0; i < actualCount; i++)
Variables.Add(variables[i]);

ArrayPool<IVariable>.Shared.Return(variables);

base.ExitInstructionExpression(expression);
}
}
}
24 changes: 0 additions & 24 deletions src/Core/Echo.Ast/Analysis/WrittenVariableFinderWalker.cs

This file was deleted.

Loading
Loading