Korean Character or UTF8 Encoding is not respected when CSV is downloaded in asp.net core

[fingers10]

Hi @JoshClose ..

Many thanks for this wonderful library. We have been using this for long time. Great work and efforts by you.

Recently we noticed that when we download csv with Korean characters and open it in Excel, we see a some symbols instead of actual Korean text. From the Github Issues, I noticed that this might be related to some encoding issue and tried adding UTF-8 Encoding but there is a difference in behaviour.

It works (meaning after saving the file when I open in excel everything looks good.) when I directly save as file as shown below

// See https://aka.ms/new-console-template for more information

using System.Globalization;
using System.Text;
using CsvHelper;
using CsvHelper.Configuration;

Console.WriteLine("Hello, World!");

var data = new List<ReportDataDto>
{
    new() { Title = "올해 부스위치가 맘에 안드시나요?" },
    new() { Title = "IFS 박람회 사무국" }
};

await GetContentBytes(data);

Console.ReadLine();
return;

async Task GetContentBytes(IEnumerable<ReportDataDto> reportData)
{
    await using var stream = new FileStream("test.csv", FileMode.Create);
    await using var sr = new StreamWriter(stream, Encoding.UTF8);
    var csvConfiguration = new CsvConfiguration(CultureInfo.InvariantCulture)
    {
        Encoding = Encoding.UTF8
    };
    await using var csv = new CsvWriter(sr, csvConfiguration);
    await csv.WriteRecordsAsync(reportData);
    await sr.FlushAsync();
}

internal class ReportDataDto
{
    [CsvHelper.Configuration.Attributes.Name("Title")]
    public string? Title { get; set; }
}

But when I use the same logic and download as csv from asp.net core and open it boom... I see symbols instead of Korean characters.

async Task<byte[]> GetContentBytes(IEnumerable<ReportDataDto> reportData)
{
    using var ms = new MemoryStream();
    await using var sr = new StreamWriter(ms, Encoding.UTF8);
    var csvConfiguration = new CsvConfiguration(CultureInfo.InvariantCulture)
    {
        Encoding = Encoding.UTF8
    };
    await using var csv = new CsvWriter(sr, csvConfiguration);
    await csv.WriteRecordsAsync(reportData);
    await sr.FlushAsync();
    return ms.ToArray();
}

//Now in controller,

var data = new List<ReportDataDto>
{
    new() { Title = "올해 부스위치가 맘에 안드시나요?" },
    new() { Title = "IFS 박람회 사무국" }
};
var dataInBytes = await GetContentBytes(data);
return File(dataInBytes, "text/csv", "test.csv");

Please can you assist me on what I'm missing?

[JoshClose]

What is ms here?

return ms.ToArray();

[fingers10]

@JoshClose Many thanks for spotting that. Its a copy paste error. ms is MemoryStream. I have now updated the code sample in the question.