Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MS Word. Manually entered abbreviations aren't preserved between Word and JurisM restarts #43

Closed
JohnLukeBentley opened this issue Mar 17, 2018 · 18 comments

Comments

@JohnLukeBentley
Copy link

On JurisM 5.0.37m10

To reproduce:

Fetch and insert a particular legal citation (an Australian, New South Wales, Statue):

  • {Browser} Surf to https://legislation.nsw.gov.au/#/view/act/2015/50.
  • {Firefox (FF)} Click on the JurisM FF connector button to save "Strata Schemes Management Act 2015 No 50 - NSW Legislation" to JurisM standalone
  • {JurisM} clean up the entry:
    • Item type to: Statue
    • "Strata Schemes Management Act 2015 No 50 - NSW Legislation" to "Strata Schemes Management Act 2015 No 50"
    • Jurisdiction: AU|New South Wales
    • Date Enacted: 2017-11-23
    • Date Published: 2018-01-14
  • {MS Word (Office 365 - Up to date)}. Open a new document.
  • {MS Word} > |Zotero| Ribbon menu > Document Properties ... > Citation Style: JM Indigo Book.
  • {MS Word} > |Zotero| Ribbon menu > Add/Edit Citation ... > "Strata" ... > Select "Strata Schemes Management Act 2015 No 50" from the dropdown box.
  • [Enter] to insert the citation in the Word doc.
  • Observe the citation:

    Strata Schemes Management Act 2015 No 50 (Australia 2017).

... all fine so far. Let's change the "Australia" Jurisdiction to the regionally specific "NSW" abbreviation ...

  • {MS Word} > Click inside previously inserted citation > |Zotero| Ribbon menu > Add/Edit Citation ... > [Abbrevs.]
  • In 'Manage Abbreviation List' > -Lists- > Choose "Place or Language" from the dropdown.
  • Against the line "AU|New South Wales [Australia]"; enter "NSW".
  • [Close]
  • Red bordered citation textbar > [Enter] to close.
  • Observe, in Word, the citation has been correctly altered to reflect the new abbreviation:

    Strata Schemes Management Act 2015 No 50 (NSW 2017).

  • {MS Word} > |Zotero| Ribbon menu > [Refresh]. Observe the citation remains unchanged (it still contains "(NSW 2017)".

... all fine so far, let's restart ...

  • Exit MS Word. Save the document if prompted.
  • Exist JurisM standalone.
  • Restart MS Word, and open the prior document.
  • Restart JurisM standalone.
  • {MS Word} > |Zotero| Ribbon menu > [Refresh].
  • Observe the citation remains changed back to ...

    Strata Schemes Management Act 2015 No 50 (Australia 2017).

... and if we go back into the "Manage Abbreviation Lists" the previously manually entered abbreviation is lost.

Looks like either a bug or I'm misunderstanding how the operation should be done.

If I am misunderstanding how the operation should be done then that might betray that the UI/workflow has room for improvement.

@JohnLukeBentley JohnLukeBentley changed the title MS Word. Manually entered abbreviations aren't preserved between Word and JurisM are restarted MS Word. Manually entered abbreviations aren't preserved between Word and JurisM restarts Mar 17, 2018
@fbennett
Copy link

Thank you for checking so carefully. This reveals a couple of bugs. I'll need to dig into it a little further, but here is what I've determined so far.

A direct inspection of the database shows that the abbreviation is actually being saved to the database when you apply the initial abbreviation. The problem is that when the document is reopened, this portion of the abbreviation data (the "raw" form of jurisdiction identifers) is being loaded incorrectly, with the result that it is not displayed among the abbreviation choices.

The "raw" key (the second column in the abbreviation listing under Places) provides a hint to the fault. Following your example steps, all three of the options for "New South Wales" show as "australia". This should not be so. I believe that the key used internally is something like "AU|New South Wales."

So (by hypothesis only, so far) if the key is correctly read (and displayed), the saved abbreviation should be available after a restart. More news on this later. I'll take a look at it today or tomorrow.

@JohnLukeBentley
Copy link
Author

Thanks for that. That's very good news that it might reveal a couple of bugs.

At this stage, being the newbie, I've found this abbreviation mechanism both awesome (in the ambition to facilitate both a standardization and user flexibility to overrule the standardization) and confusing. The confusion is in part due to my having simply to conceptually catch on to what's going on here ... but sorting out those bugs might mean that the correct entries presented to me will be less confusing.

Good luck with the debugging!

@fbennett
Copy link

There is plenty of scope for making abbrevs easier for users to understand and control. That's beyond my skill set, but others will pitch in over time, if the project persists long enough.

I've figured out one further thing about your sample. If a "Place" abbreviation is applied to a top-level country name among the jurisdictions (i.e. if you set the statute to "Australia" as the jurisdiction), the abbreviation will stick. That brings us closer to understanding the fault in the code.

As background, the jurisdiction field is stored behind the scenes as a code. Australia is "au", and New South Wales is "au:nsw". The fact that we're seeing "australia" in the listing means that "au:nsw" (which would display as "au|new south wales") is being stripped to "au" (which displays as "australia").

There are some low-level issues here that I need to think about more carefully. My initial feeling, looking at the fault and the data, is that the abbreviations database should be using the same key internally and externally as the citation data itself does internally and externally (so in your example, both should have "au:nsw" as their internal record, and "AU|New South Wales" as their displayed field value). That will make the data structures easier to follow, and allow arbitrary changes (to fix spelling and script errors in the displayed values) without breaking things.

Fixing this could affect the abbreviation behavior in existing documents that are live in the field, but it needs to be fixed, and now is a better time than later.

@JohnLukeBentley
Copy link
Author

I've added https://www.courtlistener.com/opinion/107965/brandenburg-v-ohio/ to my example above as another Entry in JurisM and another citation in my document. That is, to have the US Jurisdiction kicking around.

In {MS Word} > .... > "Manage Abbreviation LIsts" I now see, with the manual addition of the "NSW" string and the inclusion of made up column names ...

Jurisdiction Code Default Display Override Display
default australia
Australia|AU australia
AU|New South Wales australia NSW
default united states
United States|US united states

As background, the jurisdiction field is stored behind the scenes as a code. Australia is "au", and New South Wales is "au:nsw". The fact that we're seeing "australia" in the listing means that "au:nsw" (which would display as "au|new south wales") is being stripped to "au" (which displays as "australia").

Yes, that make sense to describe what I'm seeing. Given my made up column names I take it by ...

he fact that we're seeing "australia" in the listing means that "au:nsw" (which would display as "au|new south wales") is being stripped to "au" (which displays as "australia").

... you mean something like ...

The fact that we're seeing "australia" in the Default Display column of the listing means that "au:nsw" (which correctly displays as "au|new south wales" in the Jurisdiction Code column) is, in the Default Display column, being stripped to "au" (which displays as "australia" in the Default Display column).

That is, I take it you mean that the third data row should really display as

Jurisdiction Code Default Display Override Display
...
AU|New South Wales new south wales NSW
... ...

or, if you have made an alternative design choice,

Jurisdiction Code Default Display Override Display
...
AU|New South Wales nsw NSW
... ...

On UI changes (and perhaps this should be posted as a separate issue) ...

There is plenty of scope for making abbrevs easier for users to understand and control. That's beyond my skill set, but others will pitch in over time, if the project persists long enough.

Well it sounds like those couple of grad students, with that bit funding behind them, will have a good opportunity for an overhaul here.

But, short of the overhaul, there's relatively simple improvements that could be made, if my understanding of what's going in is correct. Perhaps ...

  • The [Abbrevs.] button opens the "Manage Abbreviation Lists" dialog. This seems to have document level rather than citation level scope. E.g. If I provide an Override Display abbreviation of "NSW" this will (desirably) cascade to all citations in the document having the relevant jurisdiction, not just the citation I'm currently on. Therefore that button would better live on the |Zotero| menu/ribbon. At the moment, with the [Abbrevs.] button living inside the red Add/Edit Citation box, the UI suggests any changes made will be citation specific.
  • That the scope of the abbreviation list is at the document, not citation, level might also be better conveyed by changing the title of the dialog to "Manage Abbreviation Lists for Document".
  • The addition of column headings, along the lines that I've suggested (with the exception of the word "Default" in the heading).
  • The jurisdiction codes, you've explained have an internal representation (e.g. "au:nsw") and an external display (e.g. "AU|New South Wales"). In the external display I'd suggest using a different separator, space hypen space " - ", e.g. "AU - New South Wales". That's less "computer-y" to look at, to borrow from your lexicon. Although may the pipe "|" separator is workaround for jurisdictions with hyphens in their names (?)

I'm a bit out to sea on the role of "default" rows. I see, in the JurisM standalone, a default checkbox, which presumably triggers (when ticked) the "default" row in the Abbreviations List. But I'm not seeing the use case for this. ...

... and the default button and default row abbreviation list interaction might have bug in it ...

  • In my abbreviation list I enter some overrides to make testing clearer ...

    Jurisdiction Code Default Display Override Display
    default australia JLB Australia default
    Australia|AU australia JLB Australia
    AU|New South Wales australia JLB NSW
    default united states
    United States|US united states
  • In JurisM standalone, for the Strata Schemes Management Act 2015 No 50 entry I tick the default box (against an AU|New South Wales Jurisdiction)

  • In MS Word > |Zotero| menu ribbon > [Refresh]

  • My citation looks like ..

    Strata Schemes Management Act 2015 No 50 - JLB (JLB NSW 2017).

    .. I was expecting ...

    Strata Schemes Management Act 2015 No 50 - JLB (JLB Australia default 2017).

@fbennett
Copy link

... you mean something like ...

Yes, that's right.

... that button would better live on the |Zotero| menu/ribbon.

Yes. I'd be every happy to accept a pull request if someone stepped up to make that happen.

Well it sounds like those couple of grad students, with that bit funding behind them, will have a good opportunity for an overhaul here.

I won't be asking them to touch Juris-M itself, unless they have an interest, and finish the work on jurisdiction codes early. The internals of Zotero/Juris-M have a steep and unforgiving learning curve, and don't lend themselves to short-term commitment, unfortunately.

I'm not seeing the use case for [default]

Not all references that apply abbreviations are legal references, so "default" is needed. That said, for items that to have a jurisdiction value, "default" could be dropped, that's true. I'm not at all happy with the abbreviation editing lists as they stand, but it will take awhile before we step into improving usability. Lots of potential potholes there.

.. I was expecting ...

No, that's correct. It's using the abbreviation with least distance from the jurisdiction of the item. "default" is an ultimate fallback.

@fbennett
Copy link

In the external display I'd suggest using a different separator.

I'll keep this as it stands, at least for the present. Colon works as an unambiguous delimiter in jurisdiction codes, because it's not permitted inside the elements, so the display form can be changed arbitrarily at some point down the road, so long as the underlying code is use for processing.

Vertical-bar is already used as a separator between the elements of institutional names, where it's needed because the content there is free-form text. Even if it's off-putting on first encounter, I think the barrier to getting used to it is pretty low.

@JohnLukeBentley
Copy link
Author

Thanks.

On separators. I'll just note your (reasoned) preferences on it and be silent on that topic for now.

On the abbrev button, as a ladder for someone to step up to moving it, I've opened #45.

On defaults ...

... I am seeing, thanks to your explanation, a default "ultimate fallback" in my non-legal Journal Article entry ...

Hwang, J., and K. Lee. 2014. “Determination of Outdoor Tobacco Smoke Exposure by Distance From a Smoking Source.” Nicotine & Tobacco Research 16 (4), April: 478–484. doi:https://doi.org/10.1093/ntr/ntt178.

... That is, in Word's "Manage Abbreviation List" > Title ... I see against a "default" only that title.

default       determination of outdoor tobacco smoke exposure by distance from a smoking source
default       R v Ahmad Alameddine; R v Lee McArthur [2018] NSWDC 43
Australia|AU  R v Ahmad Alameddine; R v Lee McArthur [2018] NSWDC 43
...

I have remaining confusions about defaults. But I think it would be better if I held off for now asking you about it, because we might be able to more productively exploit my ignorance on it, and some other JurisM aspects (in a manner that I won't specify here and now).

So, to summarize where we are at with this thread, there just remains the original set of bugs which you've approximately situated in the second post.

@KarlHegbloom
Copy link

KarlHegbloom commented Mar 20, 2018 via email

@fbennett
Copy link

Yes, that's right. I think the point there is just that the button opens a document-wide list of potential abbreviations, rather than one limited to the context of the citation carrying the button.

@JohnLukeBentley
Copy link
Author

I believe that abbrevs are associated with the CSL style, not with the document.

... a document-wide list of potential abbreviations ...

@KarlHegbloom and @fbennett, that changes my view of what's going on.

The "Manage Abbreviations List" does have a prominent "Style" group, within which is listed the current Style. But it also seems to be that the elements under "Lists" come from the set of citations/references that have so far been given in the document.

So would it not be more accurate to say that "Manage Abbreviation Lists" (when working as intended) stores abbreviations for the intersection of a style and a document?

@fbennett
Copy link

Yes, that would be an accurate description.

@KarlHegbloom
Copy link

KarlHegbloom commented Mar 22, 2018 via email

@KarlHegbloom
Copy link

KarlHegbloom commented Mar 22, 2018 via email

@JohnLukeBentley
Copy link
Author

@fbennett. I've updated to 5.0.39m13.

It looks like you've taken care of those Word Plug-in Abbreviation List bugs:

  • Changes are preserved between Word and JurisM restarts.
  • You've gotten rid of the superfluous "Default" row (at for the Entry Item Types I have).

An additional bug I've come across, which may or may not be related to the current set, is that the (hidden code?) jurisdiction code "us" comes through, whether you suppress the United States Jurisdiction or not.

To reproduce:

  • Surf to https://www.courtlistener.com/opinion/107965/brandenburg-v-ohio/ and save to JurisM
  • Verify entry values include:
    • Court: Supreme Court;
    • Jurisdiction: United States|US
  • {MS Word} add the Brandenburg v. Ohio citation.
  • {MS Word} > [Zotero] menu > Document preferences > JM Indigo Book.
  • {MS Word} > ... > Abbrevs.
    • Lists > "Institution Part" > "United States|US; supreme court" > Enter "XXX" to help with debugging.
  • Manage Abbreviation Lists [Close]
  • {MS Word} > [Zotero] menu > [Refresh]

I get

Brandenburg v. Ohio, 395 U.S. 444, 89 S. Ct. 1827, 23 L. Ed. 2d 430, 1969 U.S. LEXIS 1367, CourtListener (United States us XXX 1969).

I expect

Brandenburg v. Ohio, 395 U.S. 444, 89 S. Ct. 1827, 23 L. Ed. 2d 430, 1969 U.S. LEXIS 1367, CourtListener (United States XXX 1969).

Go back in the Abbreviation List and suppress the United States|US jurisdiction.

I get

Brandenburg v. Ohio, 395 U.S. 444, 89 S. Ct. 1827, 23 L. Ed. 2d 430, 1969 U.S. LEXIS 1367, CourtListener (us XXX 1969).

... in short, and to repeat, there's a rogue "us" code coming through.

By contrast this isn't appearing as a problem with my citations having Australia|NSW jurisdiction.

@fbennett
Copy link

Those steps reveal a bug, in that I am unable to complete them, at least under Linux.

Saves from CourtListener currently fail with a warning, and fall back to the Web Page type. Web Page has no "jurisdiction" field, and due to a coding error in 5.0.39m13, the item type can't be changed to anything else. Possibly your system acquired a cookie from an earlier version that enables the CourtListener translator -- I will restore the functionality in the next release.

On the country codes, that's a bug, but I get different behavior here, and United States and Australia behave in exactly the same way. In the full form, I get "United States us", and "Australia au." With suppression I get "us" and "au". The cause is in a coding error that fails to suppress the jurisdiction, and passes through the raw code value, when a top-level national jurisdiction is selected. I will fix that and make a fresh release in the afternoon.

Thanks for reporting that!

@JohnLukeBentley
Copy link
Author

Possibly your system acquired a cookie from an earlier version that enables the CourtListener translator -- I will restore the functionality in the next release.

Sorry, I had an exiting Brandenburg v. Ohio entry that I had previously imported (in an earlier version) from CourtListener and probably changed from Web Type to Case type. So if you are seeing new strange importing behaviour then I'll probably have that behaviour too (without attempting the reproduction).

In short, it sounds like you've correctly identified the bugs I'll have at my end.

Thanks for the prior and future fixes!

@fbennett
Copy link

The abbreviation failures and the CourtListener translator is properly enabled in v5.0.39m14. Lots of bugs fixed in this release -- thanks for reporting these issues!

@JohnLukeBentley
Copy link
Author

On 5.0.39m15 all these bugs look fixed at my end. Great work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants