Working with misaligned image buffers

[icebeing]

Hello all -

I'm developing a simple "image subtraction" generator class (let's call it BGSub). The "runner" code does the following steps:
1 - read in two PNG-encoded images (16-bit grayscale), with HalideTools::load_image()
2 - ensure the image data is stored in a globally-aligned buffer to some alignment value (in this case, alignment-size == 16)

the BGSub::schedule() method is written as follows:

` Var xi{"xi"}, yi{"yi"};

    //inputL, inputD are my input buffers
    inputL.dim(0).set_min(0);
    inputL.dim(1).set_min(0);
    inputD.dim(0).set_min(0);
    inputD.dim(1).set_min(0);

    auto output_buffer = Func(output).output_buffer();
    output_buffer.dim(0).set_min(0);
    output_buffer.dim(1).set_min(0);

    //LOG2_ALIGNMENT == 4; vector_size == 16
const int vector_size = 1<<LOG2_ALIGNMENT;  
    if (get_target().has_feature(Target::HVX) || vector_size > 1) {
    
        Expr inputL_stride = inputL.dim(1).stride() + vector_size-1;
    //align the y-coordinate's stride value, rounding up
        inputL.dim(1).set_stride((inputL_stride / vector_size) * vector_size);
        inputL.set_host_alignment(vector_size);

        Expr inputD_stride = inputD.dim(1).stride() + vector_size-1;	    
        inputD.dim(1).set_stride((inputD_stride / vector_size) * vector_size);
        inputD.set_host_alignment(vector_size);

        Expr output_stride = output_buffer.dim(1).stride() + vector_size-1;
        output_buffer.dim(1).set_stride((output_stride / vector_size) * vector_size);
}

Func(output).compute_root();
}

`

So, when this code tries to invoke xxx.set_stride(...) with the new value, I get this error:
Error: Constraint violated: inputL.stride.1 (1124) == ((((inputL.stride.1 + 16) - 1)/16)*16) (1136)

So, I'm wondering if the better approach is to set the desired stride value when one first decodes the PNG images? How would I do that to not bump into this crash (which seems like an assertion).

Basically; the original input image rows are not aligned to the alignment data-lane. How do I ensure this, and not crash?

Thanks, charles.

[abadams]

Those set_stride calls are asking Halide to generate assertions that the strides of the buffers in question are aligned. Halide doesn't allocate the inputs and outputs though, so it is up to the calling code to make that true.

So yes, I think you'd have to do it when decoding the PNG.

[icebeing]

OK, I have a follow-on question relating to this: I was looking at one of the example generators (Conv3x3). It uses align_storage(), as follows:

//generator() method defined earlier; bounded_input defined to use ::repeat_egde() ... ok

//in ::schedule() method
if (get_target().has_feature(Target::HVX)) {
const int vector_size = 128;
//CRM - if input is misaligned (i.e. stride-sz mod 128 != 0), won't this assert/throw?
//CRM - why are we rounding down here?
//CRM - for this to work...input has to be stride-aligned already?
Expr input_stride = input.dim(1).stride();
input.dim(1).set_stride((input_stride / vector_size) * vector_size);

        Expr output_stride = output.dim(1).stride();
        output.dim(1).set_stride((output_stride / vector_size) * vector_size);

        //CRM let me see if I understand correctly, these particular steps:
       //you're going to compute the bounded_input Func, and write it out to output?
       //the bounded_input becomes padded with the extra row zeros?
       //we vectorize along x, by 'vector_size' units (=128)
        bounded_input
            .compute_at(Func(output), y)
            .align_storage(x, 128)
            .vectorize(x, vector_size, TailStrategy::RoundUp);

I guess I just don't understand what these directives are doing, to ensure my inputs are properly aligned.

Thanks, Charles.

[abadams]

In short, if Halide allocates the memory (which is true for internal pipeline stages only - i.e. non-output Funcs with a compute_at set), then we can control the strides and you do that with align_storage. If Halide does not allocate the memory (inputs and outputs), then we can't control the strides. The best we can do is assert that they're aligned, and that's what set_stride does.

//CRM - if input is misaligned (i.e. stride-sz mod 128 != 0), won't this assert/throw?
//CRM - why are we rounding down here?
//CRM - for this to work...input has to be stride-aligned already?

and that's all correct. It's a way of asserting that the input is stride-aligned already.

[icebeing]

ok, so the takeaway here is that the responsibility of setting up an input buffer(s), meeting a certain alignment falls upon the caller/runner code. OK

So...what is the purpose of the .align_storage() directive? If my inputs already meet the required alignment, won't that always be the case for internal Func's? Or, is that useful when dispatching them to some target with a different alignment?

Cheers, charles.

[abadams]

internal Funcs are ones that aren't inputs or outputs, where the Halide pipeline is doing the allocating. E.g in the following code, f is not an input or an output, so the generated halide pipeline is responsible for allocating it and freeing it, so align_storage is meaningful:

f(x, y) = my_input(x, y) * 2;
output(x, y) = f(x / 3, y / 3) + 3;

f.compute_root().align_storage(x, 16);

In general the alignment of the input and output buffers doesn't tell you much about the alignment of intermediates. In that example, even if output happens to be a multiple of 16 wide, that doesn't tell you that the intermediate is a multiple of 16 wide.