[HOLD for payment 2023-03-13] [$1000] Whitespace above block quotes is stripped away

[kavimuru]

If you haven’t already, check out our contributing guidelines for onboarding and email [email protected] to request to join our Slack channel!

---

Action Performed:

1. Compose a message like this:

I am writing a message and it includes a quote:

> This is a quote – notice that there is a blank line above it.

See how I've typed even spacing above and below that block quote?

2. Post the message
3. Edit the message you posted

Expected Result:

There should be a blank line (and an even amount of space) above and below the block quote. The same message you sent in step 1 should be shown in the editor.

Actual Result:

There's no blank line above the block quote.
Edit message shown as

I am writing a message and it includes a quote:
> This is a quote – notice that there is a blank line above it.

See how I've typed even spacing above and below that block quote?

Workaround:

unknown

Platforms:

Which of our officially supported platforms is this issue occurring on?

• Android / native
• Android / Chrome
• iOS / native
• iOS / Safari
• MacOS / Chrome / Safari
• MacOS / Desktop

Version Number: 1.2.67-4
Reproducible in staging?: y
Reproducible in production?: y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos:

Recording.1462.mp4

Expensify/Expensify Issue URL:
Issue reported by: @roryabraham
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1675803452308849

View all open jobs on GitHub

Upwork Automation - Do Not Edit

• Upwork Job URL: https://www.upwork.com/jobs/~0162b44ca1f9c03f85
• Upwork Job ID: 1623397585262231552
• Last Price Increase: 2023-02-15

[MelvinBot]

Triggered auto assignment to @abekkala (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

[MelvinBot]

Bug0 Triage Checklist (Main S/O)

• This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
• This bug is not a duplicate report (check E/App issues and #expensify-bugs)
  • If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead
• This bug is reproducible using the reproduction steps in the OP. S/O
  • If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
  • If the reproduction steps aren't clear and you determine the correct steps, please update the OP.
• This issue is filled out as thoroughly and clearly as possible
  • Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.
• I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

[MelvinBot]

Job added to Upwork: https://www.upwork.com/jobs/~0162b44ca1f9c03f85

[MelvinBot]

Current assignee @abekkala is eligible for the External assigner, not assigning anyone new.

[MelvinBot]

Triggered auto assignment to Contributor-plus team member for initial proposal review - @rushatgabhane (External)

[MelvinBot]

Current assignee @thienlnam is eligible for the External assigner, not assigning anyone new.

[tsvetomir-marinov-dreamix]

Proposal

Please re-state the problem that we are trying to solve in this issue.

Writing whitespace above block quotes is stripped away

What is the root cause of that problem?

Looking at ExpensiMark.js seems like this is the expected behavior.

{
    name: 'quote',

    // We also want to capture a blank line before or after the quote so that we do not add extra spaces.
    // block quotes naturally appear on their own line. Blockquotes should not appear in code fences or
    // inline code blocks. A single prepending space should be stripped if it exists
    process: (textToProcess, replacement) => {
        const regex = new RegExp(
            /\n?^&gt; ?(?![^<]*(?:<\/pre>|<\/code>))([^\v\n\r]+)\n?/gm,
        );
        return this.modifyTextForQuote(regex, textToProcess, replacement);
    },
    replacement: g1 => `<blockquote>${g1}</blockquote>`,
},

What changes do you think we should make in order to solve the problem?

If we comment out this functionality the problem is resolved.

Another solution is to change the regex to capture the lines only inside quotes. Example:

>
> First Line
>

[abdulrahuman5196]

Proposal

Please re-state the problem that we are trying to solve in this issue.

New line before a block quote is removed automatically
Note: We won't be able to repro the issue in native iOS/android.

What is the root cause of that problem?

We have been intentionally removing a new line before a blockquote. For example

'<br/><blockquote>' is changed to '<blockquote>' 

here https://github.com/Expensify/expensify-common/blob/5cdfe388a759e063bd86d00440a4bf277b89a3c6/lib/ExpensiMark.js#L186-L190
This change was introduced to fix this issue - #9379 long time back.

As the regex replacement is present in the ExpensiMark.js we are removing the <br/> even before send to server for AddComment. 
But the problem is <blockquote> is behaving differently in web and differently in native iOS/android. 
In native iOS/android it automatically seems to add a new line before <blockquote> while displaying whereas in web/mweb it doesn't happen.

What changes do you think we should make in order to solve the problem?

We should remove the rule to replace br before blockquote from the ExpensiMark.js. This will fix the current issue.

But it will regress the issue #9379 it originally tried to solve. So we need to apply the regex replacement before displaying the HTML in the view for native iOS/android alone to fix the original issue.

One other reason i prefer to do this change other the alternative listed below is, if we add the issue message above in a chat and try to edit the message in native iOS/android, we would find a new line missing in the edit compose window. But it would change to normal when you save it.

We could fix both issues by the above fix.

What alternative solutions did you explore? (Optional)

I tried to explore if we could do an upstream fix in RenderHTML, but it seems there where multiple issue where people analysed on upstream fix and concluded it has to do more with rendering of blockquote at web vs native layer than the RenderHTML package scope. Also regardless of upstream fix we would be required to remove the br removal at ExpensiMark.js as its leading to the edit issue i mentioned above.
Reference
#9379
#12120
#12784
#12120

[rushatgabhane]

Hi @tsvetomir-marinov-dreamix, thanks for your proposal.

The comment you've attached says, "A single prepending space should be stripped if it exists".

Could you please help us understand why this code exists? Or to put in another way, what regression (if any) will we be causing by removing that block of code?

Thanks!

[rushatgabhane]

So we need to apply the regex replacement before displaying the HTML in the view for native iOS/android alone to fix the original issue

@abdulrahuman5196 could you please give details on how you'll achieve this? Which regex will be used and where?

[abdulrahuman5196]

@rushatgabhane could you please give details on how you'll achieve this? Which regex will be used and where?

Which regex will be used?

https://github.com/Expensify/expensify-common/blob/5cdfe388a759e063bd86d00440a4bf277b89a3c6/lib/ExpensiMark.js#L186-L190

Where to use?

We have to replace the text here before using the html -

App/src/pages/home/report/ReportActionItemFragment.js

Lines 109 to 119 in a53b274

	if (!differByLineBreaksOnly) {
	const editedTag = props.fragment.isEdited ? '<edited></edited>' : '';
	const htmlContent = html + editedTag;
	return (
	<RenderHTML
	html={props.source === 'email'
	? `<email-comment>${htmlContent}</email-comment>`
	: `<comment>${htmlContent}</comment>`}
	/>
	);
	}

We can replace the html with the above regex at other ingress before display as well, but this seems best.

[rushatgabhane]

We'll be confirming in the App whether this introduced any regressions or messed something up where we had the hack before, right

yepp. it's only visual.
im going through the code and manually verifying if the line breakers look alright on native. (random sampling)

[tienifr]

@dangrous

Why does the iOS/Android additional test work right now? Shouldn't it be messed up until we turn the flag on?

Because the flag was already added in this PR https://github.com/Expensify/App/pull/14822/files and turned on for native code so the additional tests for iOS/Android are working.
cc @rushatgabhane

[rushatgabhane]

ohhhh wtf. that's awesome

makes sense that your PR is just removing the hack. nothing less nothing more

[dangrous]

Ah! That makes much more sense.

[tienifr]

@rushatgabhane @thienlnam The PR #15504 is ready for review. Thanks

[tienifr]

@rushatgabhane Can you help review this PR #15504? Thanks

[tienifr]

Since the PR is already merged. I'm here to propose the regression test.

Regression Test Proposal

Bug: Whitespace above block quotes is stripped away

Proposed Test Steps:

For all platforms

1, Compose a message like this:

normal text

> quote text

normal text

2, Post the message
3, Edit the message you posted
4, Verify that there is a blank line (and an even amount of space) above and below the block quote. The same message you sent in step 1 is shown in the editor

Additional test for IOS and Android (to ensure there is no regression bug created by the PR)

1, Compose a message like this:

normal text
> quote text
normal text

2, Post the message
3, Verify that there is no extra space displayed when used non-quoted and quoted then non-quoted text

Do we 👍 or 👎

[MelvinBot]

Reviewing label has been removed, please complete the "BugZero Checklist".

[MelvinBot]

The solution for this issue has been 🚀 deployed to production 🚀 in version 1.2.78-0 and is now subject to a 7-day regression period 📆. Here is the list of pull requests that resolve this issue:

• Fix/14941 whitespace above block quotes is stripped away #15504

If no regressions arise, payment will be issued on 2023-03-13. 🎊

After the hold period is over and BZ checklist items are completed, please complete any of the applicable payments for this issue, and check them off once done.

• External issue reporter
• Contributor that fixed the issue
• Contributor+ that helped on the issue and/or PR

As a reminder, here are the bonuses/penalties that should be applied for any External issue:

• Merged PR within 3 business days of assignment - 50% bonus
• Merged PR more than 9 business days after assignment - 50% penalty

[MelvinBot]

BugZero Checklist: The PR fixing this issue has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

• [@rushatgabhane / @thienlnam] The PR that introduced the bug has been identified. Link to the PR:
• [@rushatgabhane / @thienlnam] The offending PR has been commented on, pointing out the bug it caused and why, so the author and reviewers can learn from the mistake. Link to comment:
• [@rushatgabhane / @thienlnam] A discussion in #expensify-bugs has been started about whether any other steps should be taken (e.g. updating the PR review checklist) in order to catch this type of bug sooner. Link to discussion:
• [@tienifr] Propose regression test steps to ensure the same bug will not reach production again.
• [@abekkala] Link the GH issue for creating/updating the regression test once above steps have been agreed upon:https://github.com/Expensify/Expensify/issues/268018

[abekkala]

@tienifr I think your proposed regression test may need some tweaking.
Also, what is the proposed location within TestRail

@rushatgabhane @thienlnam

[abekkala]

Payments to be made on MAR 13

Issue reported by: @roryabraham 🎉 no payment for you 😉
Selected Proposal for fix: @tienifr [$1000]
C+ Review: @rushatgabhane [$1000]

Accepted Proposal: Feb 22
Merged PR: Mar 02
6 Business Days

• Merged PR within 6 business days - 0% bonus

No regressions! 🎉

[abekkala]

Offer sent to @rushatgabhane

@tienifr I can't seem to find you in Upwork. Can you let me know your profile name within Upwork

[tienifr]

@abekkala I've just applied to the Upwork job, thanks

[abekkala]

Thank you @tienifr I've sent your offer!

[abekkala]

@tienifr payment sent and contract ended - thank you! 🎉

[abekkala]

@rushatgabhane payment sent and contract ended - thank you! 🎉

[abekkala]

We're all set here - closing!