Improve test coverage

[ckeshava]

The test coverage status can be estimated by running coverage html/xml/report from the root directory of the project. The below snapshot shows the stats as of July 22, 2024.

Improving coverage
While the average test coverage is at 88%, there are source files with coverage as low as 18% (xrpl/asyncio/ledger/main.py). These stats include both white-box and black-box testing of the source code. It is desirable to reach a target of 100% test coverage. This should be possible because we don't have platform/architecture-specific code in this library. This task is compatible with the OpenAPI-specification project for Client Libraries. Unit and integration tests can be used to test the machine-generated client libraries.

Github Actions
It is useful to have coverage statistics for every pull request, based on their respective diff. PR authors can be incentivized to include tests along with their changes. These reports can also be stored on the Codecov platform for future debugging and diagnosis. But, I'm not aware of the cost for the usage of such platforms.

xrpl-py-py3.11➜  xrpl-py git:(request-from-dict) ✗ coverage report
Name                                                         Stmts   Miss Branch BrPart   Cover   Missing
---------------------------------------------------------------------------------------------------------
xrpl/account/main.py                                            12      4      0      0  66.67%   30, 50, 70, 90
xrpl/account/transaction_history.py                              6      1      0      0  83.33%   23
xrpl/asyncio/account/main.py                                    24     16      6      0  26.67%   30-37, 57, 79, 104-119
xrpl/asyncio/account/transaction_history.py                     11      6      4      0  33.33%   23-30
xrpl/asyncio/clients/async_client.py                             8      1      0      0  87.50%   29
xrpl/asyncio/clients/async_websocket_client.py                  34     15     10      0  43.18%   214-215, 219-220, 229-230, 239, 248-249, 265-267, 287-289
xrpl/asyncio/clients/client.py                                  15      4      2      0  76.47%   32-34, 54
xrpl/asyncio/clients/exceptions.py                              16      8      4      0  40.00%   22-29
xrpl/asyncio/clients/json_rpc_base.py                           17      6      2      0  57.89%   43-51
xrpl/asyncio/clients/utils.py                                   32     25      4      0  19.44%   21-24, 37-48, 63-66, 79-91
xrpl/asyncio/clients/websocket_base.py                          81     52     16      1  30.93%   24-26, 45-49, 66-74, 83, 93-99, 104-119, 131-141, 156-166, 175, 191-192, 199-201, 221-237
xrpl/asyncio/ledger/main.py                                     34     25     16      0  18.00%   25-29, 45-49, 77-98
xrpl/asyncio/ledger/utils.py                                    23      1      8      1  93.55%   45
xrpl/asyncio/transaction/main.py                               154     87     68      6  34.68%   59-65, 139-142, 167-174, 232-247, 260-268, 284-289, 304-341, 361-365, 378, 392, 412-418, 450, 457-461, 468-471, 476, 482-484
xrpl/asyncio/transaction/reliable_submission.py                 61     40     26      0  24.14%   38-72, 105-116, 149-170, 206-209
xrpl/asyncio/wallet/wallet_generation.py                        80     42     36      2  41.38%   60-93, 173, 179, 185-191, 200-206, 210-217
xrpl/clients/sync_client.py                                      9      1      0      0  88.89%   31
xrpl/clients/websocket_client.py                                60     37     12      0  31.94%   83-86, 95, 99-114, 118-139, 148-149, 158, 171-185, 201-203, 226-240
xrpl/core/addresscodec/codec.py                                 71      2     14      2  95.29%   85, 128
xrpl/core/addresscodec/main.py                                  61      3     22      3  92.77%   44, 167, 169
xrpl/core/binarycodec/binary_wrappers/binary_parser.py          84     10     24      7  84.26%   16, 51, 64, 132, 168, 189, 196, 246, 260-261
xrpl/core/binarycodec/binary_wrappers/binary_serializer.py      42      1     10      1  96.15%   51
xrpl/core/binarycodec/definitions/definitions.py                59      5     14      1  91.78%   81-82, 118, 221, 234
xrpl/core/binarycodec/definitions/field_header.py               25      2      8      1  90.91%   27, 57
xrpl/core/binarycodec/definitions/field_instance.py             22      1      4      1  92.31%   14
xrpl/core/binarycodec/field_id_codec.py                         47      2     16      2  93.65%   51, 105
xrpl/core/binarycodec/types/amount.py                          133      6     55      6  93.62%   98, 137, 156, 165, 173, 176
xrpl/core/binarycodec/types/blob.py                             19      1      8      1  92.59%   64
xrpl/core/binarycodec/types/currency.py                         51      2     24      2  94.67%   82, 120
xrpl/core/binarycodec/types/hash128.py                          19      1      6      1  92.00%   49
xrpl/core/binarycodec/types/hash.py                             28      2     14      0  95.24%   41, 85
xrpl/core/binarycodec/types/issue.py                            37      1     12      1  95.92%   55
xrpl/core/binarycodec/types/path_set.py                        126      4     56      5  95.05%   55, 136, 157, 182->190, 244, 260->267
xrpl/core/binarycodec/types/serialized_type.py                  29      3     14      1  90.70%   12, 30, 35
xrpl/core/binarycodec/types/st_array.py                         46      0     18      2  96.88%   44->52, 98->106
xrpl/core/binarycodec/types/st_object.py                       103      4     58      4  95.03%   67, 78, 116, 233
xrpl/core/binarycodec/types/uint8.py                            21      1      8      1  93.10%   68
xrpl/core/binarycodec/types/uint16.py                           21      1      8      1  93.10%   66
xrpl/core/binarycodec/types/uint32.py                           24      4     10      1  79.41%   67-71
xrpl/core/binarycodec/types/uint64.py                           33      3     14      3  87.23%   69, 75, 80
xrpl/core/binarycodec/types/uint.py                             51     16     28      6  62.03%   40, 45, 48, 56, 60-64, 72, 76-80, 91
xrpl/core/binarycodec/types/vector256.py                        34      1     14      1  95.83%   79
xrpl/core/binarycodec/types/xchain_bridge.py                    43      1     20      1  96.83%   55
xrpl/core/keypairs/crypto_implementation.py                     21      3     20      0  92.68%   28, 37, 47
xrpl/core/keypairs/main.py                                      39      2      8      2  91.49%   77, 112
xrpl/core/keypairs/secp256k1.py                                 75      1     30      2  97.14%   198->191, 200
xrpl/ledger/main.py                                             10      3      0      0  70.00%   23, 39, 70
xrpl/models/amounts/amount.py                                   11      1      2      1  84.62%   51
xrpl/models/base_model.py                                      113      8     82      3  92.31%   138, 192-194, 240, 270, 341, 345-346
xrpl/models/currencies/xrp.py                                   25      3     11      2  86.11%   55, 81, 91
xrpl/models/flags.py                                            36      1     22      2  94.83%   67->65, 71
xrpl/models/nested_model.py                                     20      2      8      1  89.29%   41, 62
xrpl/models/path.py                                             40     12     23      4  61.90%   42, 44, 50-54, 59-63
xrpl/models/requests/generic_request.py                         32      4     19      3  86.27%   63-64, 68-72
xrpl/models/requests/sign_and_submit.py                         45     17     15      0  58.33%   90-95, 104-107, 110-119, 122-127
xrpl/models/requests/sign_for.py                                44     17     15      0  57.63%   72-77, 86-89, 92-101, 104-109
xrpl/models/requests/submit.py                                  18      1     11      1  93.10%   83
xrpl/models/requests/submit_multisigned.py                      25      3      9      1  88.24%   59->61, 70-72
xrpl/models/response.py                                         52      1     21      2  93.15%   97->108, 109
xrpl/models/transactions/account_set.py                        123      6     31      4  92.21%   248-250, 256, 262, 297
xrpl/models/transactions/clawback.py                            23      0     11      1  97.06%   43->46
xrpl/models/transactions/escrow_create.py                       30      0      7      1  97.30%   77->86
xrpl/models/transactions/metadata.py                            56      1      0      0  98.21%   137
xrpl/models/transactions/oracle_set.py                          69      1     34      1  98.06%   99
xrpl/models/transactions/pseudo_transactions/unl_modify.py      25      1      7      1  93.75%   69
xrpl/models/transactions/signer_list_set.py                     62      3     34      2  94.79%   102, 110-113
xrpl/models/transactions/transaction.py                        167      9     84      6  94.02%   117, 328, 347, 355-356, 397, 415, 455, 468
xrpl/models/utils.py                                            21      6      8      3  62.07%   76, 78, 86-91, 98
xrpl/transaction/main.py                                        16      4      0      0  75.00%   34, 67, 99, 126
xrpl/transaction/reliable_submission.py                          9      1      0      0  88.89%   47
xrpl/utils/get_nftoken_id.py                                    41      2     24      5  89.23%   89, 106->110, 108->110, 118->120, 121
xrpl/utils/get_xchain_claim_id.py                               10      1      8      1  88.89%   39
xrpl/utils/time_conversions.py                                  38      4     16      4  85.19%   35, 37, 89, 91
xrpl/utils/txn_parser/utils/balance_parser.py                   65      2     28      5  92.47%   28->36, 47, 106
xrpl/utils/txn_parser/utils/order_book_parser.py                98      3     38      5  94.12%   69->71, 85, 89, 113
xrpl/utils/xrp_conversions.py                                   48      5     20      1  91.18%   47-48, 64, 94-95
xrpl/wallet/main.py                                             71      4     28      3  92.93%   66, 195, 251, 283
xrpl/wallet/wallet_generation.py                                 7      1      0      0  85.71%   40
---------------------------------------------------------------------------------------------------------
TOTAL                                                         6177    581   2116    132  88.77%

146 files skipped due to complete coverage.

[ckeshava]

Working on these unit tests would be a great way to understand this codebase for new engineers

[mvadari]

Coverage checks should include both unit and integration tests. The coverage is much better in that case - 94%, with no file that matters under 70%. You can check this by running poetry run poe test_coverage.

Name                                                         Stmts   Miss Branch BrPart   Cover   Missing
---------------------------------------------------------------------------------------------------------
xrpl/account/main.py                                            12      2      0      0  83.33%   50, 90
xrpl/asyncio/account/transaction_history.py                     11      2      4      2  73.33%   24, 29
xrpl/asyncio/clients/async_websocket_client.py                  34      2     10      5  84.09%   214->exit, 219->exit, 248->exit, 266, 288
xrpl/asyncio/clients/client.py                                  15      1      2      0  94.12%   54
xrpl/asyncio/clients/exceptions.py                              16      0      4      1  95.00%   26->28
xrpl/asyncio/clients/json_rpc_base.py                           17      2      2      0  89.47%   50-51
xrpl/asyncio/clients/websocket_base.py                          81      6     16      4  89.69%   24-26, 46, 109, 163
xrpl/asyncio/ledger/main.py                                     34     11     16      4  58.00%   29, 45-49, 79, 85-90, 94->98
xrpl/asyncio/ledger/utils.py                                    23      1      8      1  93.55%   45
xrpl/asyncio/transaction/main.py                               154     26     68     11  77.03%   62-64, 262->264, 264->266, 268, 285->289, 305, 316-341, 364, 378, 392, 458->464, 469
xrpl/asyncio/transaction/reliable_submission.py                 61      7     26      6  82.76%   52-61, 113-114, 157, 161->164, 164->167, 168
xrpl/asyncio/wallet/wallet_generation.py                        80     11     36      8  81.90%   69, 86, 90->79, 93, 173, 179, 191, 206, 212-217
xrpl/clients/websocket_client.py                                60      7     12      6  81.94%   100, 119, 171->exit, 180-181, 185, 202, 227
xrpl/core/addresscodec/codec.py                                 71      2     14      2  95.29%   85, 128
xrpl/core/addresscodec/main.py                                  61      3     22      3  92.77%   44, 167, 169
xrpl/core/binarycodec/binary_wrappers/binary_parser.py          84     10     24      7  84.26%   16, 51, 64, 132, 168, 189, 196, 246, 260-261
xrpl/core/binarycodec/binary_wrappers/binary_serializer.py      42      1     10      1  96.15%   51
xrpl/core/binarycodec/definitions/definitions.py                59      5     14      1  91.78%   81-82, 118, 221, 234
xrpl/core/binarycodec/definitions/field_header.py               25      2      8      1  90.91%   27, 57
xrpl/core/binarycodec/definitions/field_instance.py             22      1      4      1  92.31%   14
xrpl/core/binarycodec/field_id_codec.py                         47      2     16      2  93.65%   51, 105
xrpl/core/binarycodec/types/amount.py                          133      6     55      6  93.62%   98, 137, 156, 165, 173, 176
xrpl/core/binarycodec/types/blob.py                             19      1      8      1  92.59%   64
xrpl/core/binarycodec/types/currency.py                         51      2     24      2  94.67%   82, 120
xrpl/core/binarycodec/types/hash128.py                          19      1      6      1  92.00%   49
xrpl/core/binarycodec/types/hash.py                             28      2     14      0  95.24%   41, 85
xrpl/core/binarycodec/types/issue.py                            37      1     12      1  95.92%   55
xrpl/core/binarycodec/types/path_set.py                        126      4     56      5  95.05%   55, 136, 157, 182->190, 244, 260->267
xrpl/core/binarycodec/types/serialized_type.py                  29      3     14      1  90.70%   12, 30, 35
xrpl/core/binarycodec/types/st_array.py                         46      0     18      2  96.88%   44->52, 98->106
xrpl/core/binarycodec/types/st_object.py                       103      4     58      4  95.03%   67, 78, 116, 233
xrpl/core/binarycodec/types/uint8.py                            21      1      8      1  93.10%   68
xrpl/core/binarycodec/types/uint16.py                           21      1      8      1  93.10%   66
xrpl/core/binarycodec/types/uint32.py                           24      4     10      1  79.41%   67-71
xrpl/core/binarycodec/types/uint64.py                           33      3     14      3  87.23%   69, 75, 80
xrpl/core/binarycodec/types/uint.py                             51     16     28      6  62.03%   40, 45, 48, 56, 60-64, 72, 76-80, 91
xrpl/core/binarycodec/types/vector256.py                        34      1     14      1  95.83%   79
xrpl/core/binarycodec/types/xchain_bridge.py                    43      1     20      1  96.83%   55
xrpl/core/keypairs/crypto_implementation.py                     21      3     20      0  92.68%   28, 37, 47
xrpl/core/keypairs/main.py                                      39      2      8      2  91.49%   77, 112
xrpl/core/keypairs/secp256k1.py                                 75      1     30      2  97.14%   198->191, 200
xrpl/ledger/main.py                                             10      1      0      0  90.00%   39
xrpl/models/amounts/amount.py                                   11      1      2      1  84.62%   51
xrpl/models/base_model.py                                      113      6     82      3  94.36%   139, 193->197, 241, 271, 342, 346-347
xrpl/models/currencies/xrp.py                                   25      3     11      2  86.11%   55, 81, 91
xrpl/models/flags.py                                            36      1     22      2  94.83%   67->65, 71
xrpl/models/nested_model.py                                     20      2      8      1  89.29%   41, 62
xrpl/models/path.py                                             40     12     23      4  61.90%   42, 44, 50-54, 59-63
xrpl/models/requests/generic_request.py                         32      1     19      1  96.08%   76
xrpl/models/requests/sign_and_submit.py                         45     17     15      0  58.33%   90-95, 104-107, 110-119, 122-127
xrpl/models/requests/sign_for.py                                44     17     15      0  57.63%   72-77, 86-89, 92-101, 104-109
xrpl/models/requests/submit.py                                  18      1     11      1  93.10%   83
xrpl/models/requests/submit_multisigned.py                      25      0      9      1  97.06%   59->61
xrpl/models/response.py                                         52      1     21      2  93.15%   97->108, 109
xrpl/models/transactions/account_set.py                        123      3     31      3  96.10%   249, 256, 297
xrpl/models/transactions/metadata.py                            56      1      0      0  98.21%   137
xrpl/models/transactions/oracle_set.py                          69      1     34      1  98.06%   99
xrpl/models/transactions/pseudo_transactions/unl_modify.py      25      1      7      1  93.75%   69
xrpl/models/transactions/signer_list_set.py                     62      3     34      2  94.79%   102, 110-113
xrpl/models/transactions/transaction.py                        167      8     84      6  94.42%   117, 328, 347, 355-356, 397, 415, 455
xrpl/models/utils.py                                            21      6      8      3  62.07%   76, 78, 86-91, 98
xrpl/utils/get_nftoken_id.py                                    41      2     24      5  89.23%   89, 106->110, 108->110, 118->120, 121
xrpl/utils/get_xchain_claim_id.py                               10      1      8      1  88.89%   39
xrpl/utils/time_conversions.py                                  38      4     16      4  85.19%   35, 37, 89, 91
xrpl/utils/txn_parser/utils/balance_parser.py                   65      2     28      5  92.47%   28->36, 47, 106
xrpl/utils/txn_parser/utils/order_book_parser.py                98      3     38      5  94.12%   69->71, 85, 89, 113
xrpl/utils/xrp_conversions.py                                   48      5     20      1  91.18%   47-48, 64, 94-95
xrpl/wallet/main.py                                             71      4     28      3  92.93%   66, 195, 251, 283
---------------------------------------------------------------------------------------------------------
TOTAL                                                         6180    268   2116    165  94.01%

[ckeshava]

I see, okay, this makes sense.