Clean up XML template files #23

duskybomb · 2018-05-26T08:45:10Z

Let's discuss it here.
Issue #22

It is validating against XML schema and passes.
able to print xml using write_xml()
unable to print pdf using write_pdf because some fields are conflicting. Example I have replace date data with '0' but white printing it cannot convert it to date format.

duskybomb · 2018-05-27T07:22:17Z

What should we do about this?

m3nu · 2018-05-27T07:25:46Z

I see you implemented a function to remove existing XML data from sample files. I don't think this is transparent enough since I can't easily look at the empty XML files and validate they are empty.

It would be better to split the current XML files into 2 folders. One for samples (to view and develop with) and one for empty templates.

factur-x/facturx/flavors/[zugferd|facturx|ubl]/xml/ (has empty XML templates)
factur-x/facturx/flavors/[zugferd|facturx|ubl]/xml/samples (has current sample files)

duskybomb · 2018-05-27T07:29:11Z

Okay, so this way we don't need to create any function and can just copy empty XML templates.

m3nu · 2018-05-27T07:30:47Z

Yes. I think this is the most transparent solution. So we can just look at the empty XML and validate it. In the accounting industry, no mistake can happen. Like forgetting to remove some old value from an XML template. That's what I meant from the start. From the way you implemented it, I see now that I wasn't clear enough on the implementation step.

duskybomb · 2018-05-27T07:44:05Z

Like this?

<?xml version="1.0" encoding="UTF-8"?>
<rsm:CrossIndustryInvoice xmlns:qdt="urn:un:unece:uncefact:data:standard:QualifiedDataType:100" xmlns:ram="urn:un:unece:uncefact:data:standard:ReusableAggregateBusinessInformationEntity:100" xmlns:rsm="urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100" xmlns:udt="urn:un:unece:uncefact:data:standard:UnqualifiedDataType:100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <rsm:ExchangedDocumentContext>
        <ram:BusinessProcessSpecifiedDocumentContextParameter>
            <ram:ID/>
        </ram:BusinessProcessSpecifiedDocumentContextParameter>
        <ram:GuidelineSpecifiedDocumentContextParameter>
            <ram:ID/>
        </ram:GuidelineSpecifiedDocumentContextParameter>
    </rsm:ExchangedDocumentContext>
    <rsm:ExchangedDocument>
        <ram:ID/>
        <ram:TypeCode>0</ram:TypeCode>
        <ram:IssueDateTime>
            <udt:DateTimeString format="102">00000000</udt:DateTimeString>
        </ram:IssueDateTime>
    </rsm:ExchangedDocument>
    <rsm:SupplyChainTradeTransaction>
        <ram:ApplicableHeaderTradeAgreement>
            <ram:BuyerReference/>
            <ram:SellerTradeParty>
                <ram:Name/>
                <ram:SpecifiedLegalOrganization>
                    <ram:ID schemeID="0002">0</ram:ID>
                </ram:SpecifiedLegalOrganization>
                <ram:PostalTradeAddress>
                    <ram:CountryID/>
                </ram:PostalTradeAddress>
                <ram:SpecifiedTaxRegistration>
                    <ram:ID schemeID="VA"/>
                </ram:SpecifiedTaxRegistration>
            </ram:SellerTradeParty>
            <ram:BuyerTradeParty>
                <ram:Name/>
                <ram:SpecifiedLegalOrganization>
                    <ram:ID schemeID="0002">0</ram:ID>
                </ram:SpecifiedLegalOrganization>
            </ram:BuyerTradeParty>
            <ram:BuyerOrderReferencedDocument>
                <ram:IssuerAssignedID/>
            </ram:BuyerOrderReferencedDocument>
        </ram:ApplicableHeaderTradeAgreement>
        <ram:ApplicableHeaderTradeDelivery/>
        <ram:ApplicableHeaderTradeSettlement>
            <ram:InvoiceCurrencyCode/>
            <ram:SpecifiedTradeSettlementHeaderMonetarySummation>
                <ram:TaxBasisTotalAmount currencyID="">0.0</ram:TaxBasisTotalAmount>
                <ram:TaxTotalAmount currencyID="">0.0</ram:TaxTotalAmount>
                <ram:GrandTotalAmount currencyID="">0.0</ram:GrandTotalAmount>
                <ram:DuePayableAmount currencyID="">0.0</ram:DuePayableAmount>
            </ram:SpecifiedTradeSettlementHeaderMonetarySummation>
        </ram:ApplicableHeaderTradeSettlement>
    </rsm:SupplyChainTradeTransaction>
</rsm:CrossIndustryInvoice>

duskybomb · 2018-05-28T05:43:13Z

I haven't tested all but minimum.xml. And it works just fine.

m3nu · 2018-06-02T11:32:51Z

facturx/flavors/factur-x/xml/basic.xml

        <ram:IssueDateTime>
-            <udt:DateTimeString format="102">20171031</udt:DateTimeString>
+            <udt:DateTimeString format="102">00000000</udt:DateTimeString>


Why so many 00 and not an empty tag?

Because dates have 8 digits?

Why not use an empty tag, like you did with all other tags? You said that's good practice. Why add placeholders for this one?

m3nu · 2018-06-02T11:33:44Z

facturx/flavors/factur-x/xml/basic.xml

@@ -1,165 +1,165 @@
 <rsm:CrossIndustryInvoice xmlns:qdt="urn:un:unece:uncefact:data:standard:QualifiedDataType:100" xmlns:ram="urn:un:unece:uncefact:data:standard:ReusableAggregateBusinessInformationEntity:100" xmlns:rsm="urn:un:unece:uncefact:data:standard:CrossIndustryInvoice:100" xmlns:udt="urn:un:unece:uncefact:data:standard:UnqualifiedDataType:100" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <rsm:ExchangedDocumentContext>
        <ram:BusinessProcessSpecifiedDocumentContextParameter>
-            <ram:ID>A1</ram:ID>
+            <ram:ID/>


Will lxml expand the tag again? I would have expected to see <ram:ID></ram:ID> for an empty tag.

Yes, the tags are expanded when any data is added. It's good practice to have empty tags as <tag/> rather than <tag></tag>.

If they are expanded it's fine.

m3nu · 2018-06-02T11:34:51Z

Please provide a minimal example that I can run to see that the empty XML templates can be filled again. Can post it here as comment.

duskybomb · 2018-06-02T12:07:36Z

You can try this to check:

from facturx import *
factx = FacturX('invoice.pdf')  # where invoice.pdf doesn't contain any embedded metadata
factx['seller'] = 'Manu'

m3nu · 2018-06-03T08:22:42Z

facturx/flavors/factur-x/xml/basic.xml

        </ram:IncludedNote>
    </rsm:ExchangedDocument>
    <rsm:SupplyChainTradeTransaction>
        <ram:IncludedSupplyChainTradeLineItem>
            <ram:AssociatedDocumentLineDocument>
-                <ram:LineID>1</ram:LineID>
+                <ram:LineID>0</ram:LineID>


This can't be 0. It's the line numbering.

m3nu · 2018-06-03T08:24:35Z

facturx/flavors/factur-x/xml/basic.xml

-                    <ram:CityName>PARIS</ram:CityName>
-                    <ram:CountryID>FR</ram:CountryID>
+                    <ram:PostcodeCode>0</ram:PostcodeCode>
+                    <ram:LineOne/>


How did you decide on using an empty tag vs. 0 as placeholder?

m3nu · 2018-06-03T08:25:51Z

facturx/flavors/factur-x/xml/basicwl.xml

        </ram:IncludedNote>
        <ram:IncludedNote>
-            <ram:Content>RCS NANTERRE 999 888 777</ram:Content>


When there are multiple invoice notes, you can't just make them all empty. We only keep one. Our current logic doesn't even allow having multiple elements at the same path.

m3nu · 2018-06-03T08:28:08Z

Still need major changes before merging. See comments and

you didn't show a full understanding of the XML document. So you are not sure how to clean it up. Having a better understanding of the document will be helpful for you.
After you empty fields, you need to see if a) we have them covered as editable fields, b) they are useless, c) they could have a default value that never changes.

duskybomb · 2018-06-03T10:32:48Z

Can you provide me some resource from where I can understand the XML document?

m3nu · 2018-06-03T17:44:15Z

The XML represents an invoice. You can look at the sample invoice PDF content and see how they relate to the XML. Then try again.

duskybomb · 2018-06-05T05:51:20Z

Here are the things I have gathered till know:

Only four fields can be changed using pdfwriter.py
- seller
- invoice_number
- date
- doc_type
Since only four fields can be changed using python, we can extend it to at least change _required fields. And as most of the fields needs to be changed manually we should use <tag></tag> instead of <tag/> for empty tags.
BusinessProcessSpecifiedDocumentContextParameter is something that is not related to invoice and maybe useless or can have a default value.
GuidelineSpecifiedDocumentContextParameter/ID needs to have a default value for every level. Like for minimum it is urn:factur-x.eu:1p0:minimum

m3nu · 2018-06-05T17:55:06Z

I don't know what you mean with your last comment. The stuff you write about the "only four fields can be changed" is simply not true. Maybe you want to check again.

m3nu · 2018-06-05T17:57:49Z

It seems you are mixing up the module for creating the PDF with the module responsible for the XML document. That's totally unrelated...

* Assigning a placeholder if some restriction is there in XSD * Or if it contains default value * All the templates have been successfully validated against XSD

m3nu · 2018-06-07T17:22:53Z

facturx/flavors/factur-x/xml/basic.xml

        <ram:TypeCode>380</ram:TypeCode>
        <ram:IssueDateTime>
-            <udt:DateTimeString format="102">20171031</udt:DateTimeString>
+            <udt:DateTimeString format=""></udt:DateTimeString>


We need to keep the default time format. Else we need to code support for different formats which isn't practical.

So should I leave all the attributes as is?

leave all the time formats. format="102". Else we need to take care of time formatting in many places.

m3nu · 2018-06-07T17:25:14Z

It seems like you didn't adjust the code to use the new empty templates you created. That would be here.

duskybomb · 2018-06-07T17:30:46Z

I don't think there is any need to adjust the code, because the old templates are moved to sample and the new empty templates have been added in-place of older ones.

First commit to clean up XML template files

0c4dbe5

duskybomb mentioned this pull request May 27, 2018

Clean up XML template files #22

Open

Clean xml file without making any function (factur-x)

cf5853b

duskybomb changed the title ~~First commit to clean up XML template files~~ Clean up XML template files May 29, 2018

m3nu reviewed Jun 2, 2018

View reviewed changes

m3nu closed this Jun 2, 2018

m3nu reopened this Jun 2, 2018

m3nu reviewed Jun 3, 2018

View reviewed changes

Cleaned Files after review

8a6e997

* Assigning a placeholder if some restriction is there in XSD * Or if it contains default value * All the templates have been successfully validated against XSD

m3nu reviewed Jun 7, 2018

View reviewed changes

retain time format

98b3ff8

duskybomb added 2 commits June 7, 2018 23:38

Merge branch 'master' into issue/22

5be6174

add a blank line

760f5d5

m3nu merged commit 26cf6c9 into master Jun 7, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up XML template files #23

Clean up XML template files #23

duskybomb commented May 26, 2018

duskybomb commented May 27, 2018

m3nu commented May 27, 2018 •

edited

Loading

duskybomb commented May 27, 2018

m3nu commented May 27, 2018

duskybomb commented May 27, 2018

duskybomb commented May 28, 2018

m3nu Jun 2, 2018

duskybomb Jun 2, 2018 •

edited

Loading

m3nu Jun 2, 2018

m3nu Jun 2, 2018

duskybomb Jun 2, 2018

m3nu Jun 2, 2018

m3nu commented Jun 2, 2018

duskybomb commented Jun 2, 2018

m3nu Jun 3, 2018

m3nu Jun 3, 2018

m3nu Jun 3, 2018

m3nu commented Jun 3, 2018

duskybomb commented Jun 3, 2018

m3nu commented Jun 3, 2018 •

edited

Loading

duskybomb commented Jun 5, 2018

m3nu commented Jun 5, 2018

m3nu commented Jun 5, 2018

m3nu Jun 7, 2018

duskybomb Jun 7, 2018

m3nu Jun 7, 2018

m3nu commented Jun 7, 2018

duskybomb commented Jun 7, 2018

Clean up XML template files #23

Clean up XML template files #23

Conversation

duskybomb commented May 26, 2018

duskybomb commented May 27, 2018

m3nu commented May 27, 2018 • edited Loading

duskybomb commented May 27, 2018

m3nu commented May 27, 2018

duskybomb commented May 27, 2018

duskybomb commented May 28, 2018

Choose a reason for hiding this comment

duskybomb Jun 2, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m3nu commented Jun 2, 2018

duskybomb commented Jun 2, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m3nu commented Jun 3, 2018

duskybomb commented Jun 3, 2018

m3nu commented Jun 3, 2018 • edited Loading

duskybomb commented Jun 5, 2018

m3nu commented Jun 5, 2018

m3nu commented Jun 5, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

m3nu commented Jun 7, 2018

duskybomb commented Jun 7, 2018

m3nu commented May 27, 2018 •

edited

Loading

duskybomb Jun 2, 2018 •

edited

Loading

m3nu commented Jun 3, 2018 •

edited

Loading