Skip to content

Releases: MobilityData/gtfs-validator

v6.0.0

25 Oct 16:24
@davidgamez davidgamez
v6.0.0
9c5c6af
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v6.0.0 Latest
Latest

Highlights

Partial validation support for Flex

GTFS Flex is a proposal that ensures riders can discover demand responsive services and how to book them. Flex was officially adopted into GTFS this March. With this validator release, we offer partial validation support for feeds that include Flex data. This includes:

summary

  • No more false positives! Before, feeds with Flex features would trigger certain notices when the data was actually valid, such as missing_required_field for a missing stop_id in stop_times.txt when location_group_id or location_id was provided. Now, any error or warning generated for your Flex feed is legitimate and should be fixed.

  • Validation for booking_rules.txt. 10 new validation notices have been added to check data from booking_rules.txt

  • Validation for polygons in locations.geojson. Checks that each polygon in locations.geojson conforms to the OpenGIS Simple Features Specification, section 6.1.11, as required by GTFS.

We'll continue to add more Flex validation rules in future releases. You can see the full list of validation rules here.

Notice changes

  • 🔴 missing_required_file: feed_info.txt is now conditionally required when translations.txt exists. Clarification adopted in google/transit#460.
  • 🔴 missing_required_field for transfers.txt: from_stop_id and to_stop_id are no longer required for a transfer_type of 4 or 5. Clarification adopted in google/transit#455.
  • 🔴 foreign_key_violation: Now checks if network_id exists in fare_leg_rules.txt but not in routes.txt OR networks.txt. Before, there was a bug that meant network_id had to be included in both routes.txt AND networks.txt.
  • 🟡 missing_timepoint_value: Modified to clarify that timepoint values should be set explicitly to 0 or 1, not empty. Clarification adopted into GTFS in google/transit#474. 🟡 missing_recommended_column was also removed as a result of this clarification.

Flex notice changes

  • 🔴 missing_required_file: stops.txt is now conditionally required. If locations.geojson is not present, stops.txt is required.
  • 🔴 missing_required_field: stop_id is conditionally required in stop_times.txt, in cases where the record does not include location_id or location_group_id.
  • 🔴 missing_trip_edge: Only checks stop times records if they do not include start_pickup_dropoff_window or end_pickup_drop_off_window.
  • 🔴 decreasing_or_equal_stop_time_distance: Skips stop times records with a location_id or location_group_id instead of a stop_id. Cases where shape_dist_traveled is provided with a location_id or location_group_id are triggered in the new 🔴 forbidden_shape_dist_traveled error.
  • 🟡 stop_without_stop_time: Skips stops that are a part of a location group referenced in stop_times.txt.
  • 🟡 stop_too_far_from_shape: Only checks stop times records with a stop_id.

New notices

  • 🔴 bidirectional_exit_gate: Triggered when exit gates (pathway_mode=7 in pathways.txt) are bidirectional.
  • 🔴 invalid_character: Triggered when text contains invalid characters, such as the replacement character ("�"). Check that text was properly encoded in UTF-8 as required by GTFS.
  • 🟡 single_shape_point: Triggered when a shape in shapes.txt only includes one shape point.

Flex new notices

  • 🔴 forbidden_shape_dist_traveled: Triggered when shape_dist_traveled is provided with a location_id or location_group_id in stop_times.txt.
  • 🔴 forbidden_geography_id: Triggered when a stop_times.txt includes more than one geography id. Only one of stop_id, location_group_id or location_id can be defined for a given entry.

locations.geojson notices

  • 🔴 malformed_json: Triggered when locations.geojson is malformed.
  • 🔴 invalid_geometry: Triggered when polygon in locations.geojson is unparsable or invalid. Each polygon must be valid by the definition of the OpenGIS Simple Features Specification, section 6.1.11.
  • 🔴 missing_required_element: Triggered when an element that's required in the geoJSON file is missing.
  • 🔴 unsupported_geo_json_type: Triggered when geoJSON type is a value other than "FeatureCollection".
  • 🔴 unsupported_feature_type: Triggered when feature type is a value other than "Feature".
  • 🔴 unsupported_geometry_type: Triggered when the geometry type is a value other than "Polygon" or "MultiPolygon".

booking_rules.txt notices

  • 🔴 forbidden_prior_day_booking_field_value: Triggered when a forbidden field value is present for a prior-day booking rule.
  • 🔴 forbidden_prior_notice_start_day: prior_notice_start_day value is forbidden when prior_notice_duration_max is set.
  • 🔴 forbidden_prior_notice_start_time: prior_notice_start_time value is forbidden when prior_notice_start_day value is not set
  • 🔴 forbidden_real_time_booking_field_value: A forbidden field value is present for a real-time booking rule.
  • 🔴 forbidden_same_day_booking_field_value: A forbidden field value is present for a same-day booking rule.
  • 🔴 invalid_prior_notice_duration_min: The prior_notice_duration_max field value needs to be greater or equal to the prior_notice_duration_min field value.
  • 🔴 missing_prior_notice_duration_min: prior_notice_duration_min value is required for same day booking_type
  • 🔴 missing_prior_day_booking_field_value: prior_notice_last_day and prior_notice_last_time values are required for prior day booking_type.
  • 🔴 missing_prior_notice_start_time: prior_notice_start_time value is required when prior_notice_start_day value is set
  • 🔴 prior_notice_last_day_after_start_day: prior_notice_last_day should not be greater than the prior_notice_start_day.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

New Contributors

PR List

  • feat: Changed notice migration generation. Added zip file to release. by @jcpitre in #1736
  • feat: documented the changes made in PR 1633 by @qcdyx in #1748
  • Feat: 1639 release process for minor versions by @jcpitre in #1657
  • feat: notify mobility-feeds-api when the web validator is updated by @cka-y in #1760
  • fix: set transfer from and to stop ids as conditional required fields by @davidgamez in #1772
  • feat: add performance assessment to acceptance tests by @cka-y in #1771
  • feat: New rule - SingleShapePointValidator for flagging shapes with a single shape point by @praneethd7 in #1753
  • Docs: Fix typo in FEATURES.md by @isabelle-dr in #1777
  • fix: Display empty values for feed_start_date and feed_end_date if there's… by @qcdyx in #1798
  • fix: Count appears as 1 for block even when no blocks exist by @qcdyx in #1802
  • fix: Added a validator for bad networkId foreign key in fareLegRules by @jcpitre in #1804
  • feat: added new field for rule priority in GtfsFareLegRuleSchema by @qcdyx in #1806
  • feat: 1791 feed infotxt should be added to missing required file when translationstxt exists by @qcdyx in #1803
  • feat: 1786 exit gates pathway mode7 must not be bidirectional by @qcdyx in https://github....
Read more

Contributors

davidgamez, qcdyx, and 6 other contributors
Assets 12
Loading

v5.0.1

20 Mar 21:46
@jcpitre jcpitre
v5.0.1
3fc961b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v5.0.1

This minor release was created to fix a key bug that was preventing feed URLs from running successfully in the Web UI. This bug only impacted the Web UI, not the desktop app or CLI tool.

For more details on our 5.0.0 release, you can read about them here.

PR List

  • chore: remove credentials on ui build by @davidgamez in #1700
  • chore(deps-dev): bump follow-redirects from 1.15.5 to 1.15.6 in /web/client by @dependabot in #1717
  • chore(deps-dev): bump vite from 4.3.9 to 4.5.2 in /web/client by @dependabot in #1705
  • chore(deps-dev): bump word-wrap from 1.2.3 to 1.2.5 in /web/client by @dependabot in #1697
  • docs: Automated update of NOTICE_MIGRATION.md by @github-actions in #1716
  • hotfix: jobId incorrectly set in the UI by @cka-y in #1723

Full Changelog: v5.0.0...v5.0.1

Contributors

davidgamez, dependabot, and cka-y
Assets 12
Loading

v5.0.0

15 Mar 00:55
@jcpitre jcpitre
v5.0.0
4029a37
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v5.0.0

Highlights

Reducing the noisiness of errors

There have been repeated issues with the sensitivity of
trip_distance_exceeds_shape_distanceand equal_shape_distance_diff_coordinates. No more! We did an extensive review of real life GTFS data from the Mobility Database and decided on thresholds for these two errors. Now data producers will only see these errors when they’re a significant issue for data quality. Below the thresholds, these issues will generate a warning instead.

Breaking changes: JSON report improvements

  • Issue #1652: Use ISO_ZONED_DATE_TIME for validatedAt field to be machine readable.
  • Issue #1685 Change gtfsComponents label to gtfsFeatures.

Notice changes

  • equal_shape_distance_diff_coordinates: Error is triggered when there's 1.11 metres or more distance between the different shape points. Discussion here.
  • trip_distance_exceeds_shape_distance: Error is triggered when there's 11.1 metres or more distance between the last stop point and the last shape point. Discussion here.
  • stop_without_zone_id changed from 🔴 ERROR to ⚪ INFO and updated so it is only triggered when there are route-based fare related fields in fare_rules.txt . Discussed in the spec in google/transit#432.

New notices

  • route_networks_specified_in_more_than_one_file: New 🔴 ERROR for Fares v2 data. Added to spec in google/transit#405.
  • equal_shape_distance_diff_coordinates_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the different shape points is less than 1.11 metres.
  • trip_distance_exceeds_shape_distance_below_threshold: New 🟡 WARNING that is triggered when the distance between the last stop point and the last shape point is less than 11.1 metres.

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility Database in this spreadsheet.

Other notable improvements

New Contributors

PR List

  • docs: Update NOTICE_MIGRATION.md by @emmambd in #1599
  • Docs: Add mention to update version number on gtfs.org by @isabelle-dr in #1600
  • feat: Support injection of arbitrary types via ValidationContext by @aababilov in #1607
  • fix: Add label to step that creates an issue by @fredericsimard in #1610
  • fix: cypress error scenario on downloading rules definitions file by @davidgamez in #1617
  • feat: Update references to Dataset Publishing Guidelines from #1601 by @emmambd in #1609
  • Update release process so we run analytics before release is launched by @emmambd in #1612
  • fix: validator version displayed in validator report summary by @jcpitre in #1627
  • fix: Correct links to rule documentation by adding -rule by @bradyhunsaker in #1629
  • fix: decreasing_or_equal_stop_time_distance includes prevShapeDistTraveled by @cka-y in #1631
  • feat: #1619 Added a bit of documentation to refer to gtfs-validator-example. by @jcpitre in #1635
  • fix: Feed generates error to authorize upload by @davidgamez in #1633
  • refactor: Replace CurrentDateTime with DateForValidation and use LocalDate by @bradyhunsaker in #1636
  • docs: Update PULL_REQUEST_TEMPLATE.md by @emmambd in #1622
  • fix: authorize upload error on create job endpoint by @davidgamez in #1642
  • fix: Corrected a problem where the version was null in the desktop version. by @jcpitre in #1643
  • feat: Add an argument for the cli (--date) that specifies the date to use during verification. by @bradyhunsaker in #1628
  • fix: Corrected a problem where code was not multithread safe by @jcpitre in #1653
  • Feat: 1640 add remaining gtfs features to the validator by @qcdyx in #1656
  • feat: removed route name and agency information components by @qcdyx in #1659
  • Minor documentation changes by @jcpitre in #1658
  • feat: added GtfsNetworkSchema by @qcdyx in #1660
  • feat: Added features document. Some corrections to components presence by @jcpitre in #1662
  • fix: prevent exception raised when serviceId is missing from calendar file by @davidgamez in #1646
  • Fix: replace GitRunners with GitHub's own runners by @fredericsimard in #1664
  • Fix: replaced Gitrunners with GitHub's own servers by @fredericsimard in #1673
  • fix: reformatted doc using HTML table format by @qcdyx in #1669
  • fix: update gradle wrapper by @qcdyx in #1677
  • feat: generated CLI package installer by @cka-y in #1680
  • feat: filter out acceptance tests from validator usage report by @cka-y in #1683
  • chore: add skip version check to the master build by @davidgamez in #1684
  • feat: validation rules for networks and route_networks by @cka-y in #1671
  • fix: cypress tests pointing to production are triggered on PRs checks by @davidgamez in #1687
  • feat: 1651 web validator retries with bad zip file 2 by @qcdyx in #1686
  • fix: table not properly rendered in notices by @cka-y in #1695
  • feat: used the standard date time formatter by @qcdyx in #1691
  • feat: replace GTFS Components language to GTFS features by @cka-y in #1696
  • feat: threshold of 1.11m on equal_shape_distance_diff_coordinates by @cka-y in #1675
  • fix: Added a execution result file by @jcpitre in #1690
  • feat: reformat trip and shape dist validator by @cka-y in #1676
  • feat: display ERRORS first on validator documentation by @cka-y in #1667
  • fix: make stop_without_zone_id conditional on fare rule type (#1663) by @michaelandrewkearney in #1693
  • fix: correct field name in missing_recommended_field_notice (#1574) by @michaelandrewkearney in #1706
  • fix: 1707 web validator waits for validation forever by @jcpitre in #1711
  • docs: update deprecated architecture information by @michaelandrewkearney in #1694
  • fix: setting package installer java version to 17.0 by @cka-y in #1709
  • feat: 1689 web validator app consumes execution results json by @davidgamez in #1712
  • feat: Add examples table for mixed_case_recommended_field by @emmambd in #1708

Full Changelog: v4.2.0...v5.0.0

Contributors

aababilov, davidgamez, and 8 other contributors
Assets 12
Loading

v4.2.0

07 Nov 17:13
@jcpitre jcpitre
v4.2.0
fdd6c9b
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.2.0

Overview

This release includes

  • Support for time variable fares that was adopted in GTFS in July 2023
  • A new summary section of the validation report, including tags to see if a feed has GTFS components like Blocks, Frequencies, or Fares v2
  • Automatic generation of the RULES documentation on https://gtfs-validator.mobilitydata.org/rules.html
  • New releases published to Maven

New notices

Notices for Fares v2 - time variable fares, added to the spec in google/transit#357

  • timeframe_overlap 🔴 ERROR
  • timeframe_only_start_or_end_time_specified 🔴 ERROR
  • timeframe_start_or_end_time_greater_than_twenty_four_hours 🔴 ERROR

Changed notices

  • Downgraded platform_without_parent_station 🟠WARNING → ⚪ INFO
  • Replaced missing_timepoint_column 🟠WARNING → missing_recommended_column 🟠WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

Full Changelog: v4.1.0...v4.2.0

New Contributors

What's Changed

Rules

  • fix: Downgraded platform_without_parent_station notice to info by @jcpitre in #1444
  • feat: Add a rule that stations (location_type 1) must be the parent_station of some stop (location_type 0). by @bradyhunsaker in #1493
  • feat: Added missing_recommended_column notice by @jcpitre in #1470
  • feat: Validation for timeframes.txt by @bdferris-v2 in #1518
  • feat: 1423 only warn if feed contact email or url are both unset by @qcdyx in #1514
  • feat: validator for ferry trips + bikes allowance by @cka-y in #1510
  • feat: 1504 add new error if a gtfs dataset has gtfs files in sub folders by @qcdyx in #1535
  • feat: trip vs shape distance validation by @cka-y in #1553
  • fix: comments on #1587 by @cka-y in #1592

Bug fixes

New summary section of report

Generate documentation automatically

  • fix: Refactor MainTest to avoid stubbing System.exit() by @bdferris-v2 in #1410
  • feat: Add @GtfsValidationNotice annotations to all notice classes by @bdferris-v2 in #1366
  • feat: Add unit-test check for @GtfsValidationNotice annotation and notices that I missed by @bdferris-v2 in #1413
  • feat: Include section references in the notice output schema. by @bdferris-v2 in #1416
  • feat: Add Notice documentation header check. by @bdferris-v2 in #1422
  • feat: Change Notice field comment format and unit-test to enforce they exist by @bdferris-v2 in #1421
  • Fix: Switch javadoc aggregation plugin to non-jar version. by @bdferris-v2 in #1432
  • feat: Add a few missing "Affected files" references by @bdferris-v2 in #1427
  • fix: Use correct import for @nullable annotation. by @bdferris-v2 in #1438
  • Feat: Bulk update Notice Javadoc to include content from RULES.md. by @bdferris-v2 in #1428
  • feat: Add documentation update + unit-test for new notice doc conventions. by @bdferris-v2 in #1429
  • feat: Split the notice doc comment into a short summary string and additional documentation. by @bdferris-v2 in #1452
  • feat: Additional test to enforce documentation comment conventions by @bdferris-v2 in #1450
  • Fix: bug in Notice documentation comment splitting. by @bdferris-v2 in #1479
  • feat: Resolve notice severity level from @GtfsValidationNotice annotation. by @bdferris-v2 in #1475
  • Feat: Remove SeverityLevel from ValidationNotice constructor by @bdferris-v2 in #1489
  • feat: Remove unit-test that RULES.md is up-to-date with Notice classes. by @bdferris-v2 in #1517
  • Remove unit-test that RULES.md is up-to-date with Notice classes by @qcdyx in #1522

Web-based validator improvements

  • feat: integrate Cypress for E2E testing by @mijogu in #1396
  • chore: ignore web packages on acceptance and e2e tests by @davidgamez in #1403
  • feat: report link sharing by @...
Read more

Contributors

briandonahue, Whoops, and 17 other contributors
Assets 12
Loading

v4.1.0

26 Apr 21:43
@isabelle-dr isabelle-dr
v4.1.0
80550f3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.1.0

Overview

This release includes the addition of the Web version available at https://gtfs-validator.mobilitydata.org/, new notices, support for Fare Media that was adopted in GTFS in March 2023, and initial support to programmatically generate the notice documentation.
See the notice mapping between releases in NOTICE_MIGRATION.md.

New notices

  • invalid_currency_amount 🔴 ERROR
  • missing_stop_name 🔴 ERROR
  • stop_without_location 🔴 ERROR
  • too_many_rows 🔴 ERROR
  • duplicate_fare_media 🟠 WARNING
  • expired_calendar 🟠 WARNING
  • missing_feed_info_date 🟠 WARNING
  • mixed_case_recommended_field 🟠 WARNING
  • route_long_name_contains_short_name 🟠 WARNING
  • trip_coverage_not_active_for_next7_days 🟠 WARNING

Impacts on existing data

You can find the impact of this release on all existing datasets we have in the Mobility database in this spreadsheet.

What's Changed

Web-based validator

Bug fixes

Rules

Documentation updates

Generate the doc programmatically

  • feat: Initial entry for @GtfsValidationNotice annotation by @bdferris-v2 in #1361
  • feat: Update Notice source code such that field name comments match documentation in RULES.md by @bdferris-v2 in #1346
  • feat: Add unit-test to check consistency of Notice field names. by @bdferris-v2 in #1345
  • fix: Inconsistencies in RULES.md against underlying source by @bdferris-v2 in #1341
  • feat: Add unit test to verify that every Notice has an entry in RULES.md by @bdferris-v2 in #1331
  • feat: Extract notice docs from source-code for in-app use and schema export by @bdferris-v2 in #1365

Acceptance tests improvements

  • feat: Update acceptance test report to include changes in WARNINGS. by @bdferris-v2 in #1354
  • feat: Fail acceptance tests on significant change in warnings by @bdferris-v2 in #1377

General maintenance + other good stuff

  • Use smaller integer field types for GTFS entity classes by @aababilov in #1273
  • Introduce GtfsEntityBuilder interface by @aababilov in #1286
  • Use int instead of long to keep CSV row number by @aababilov in #1287
  • Generate registries of tables and validators by @aababilov in #1290
  • Avoid unchecked conversion in table and validator registries by @aababilov in #1293
  • build(deps): bump certifi from 2021.5.30 to 2022.12.7 in /scripts/mobility-database-harvester by @dependabot in #1299
  • Separate table loading logic from table structure by @aababilov in #1284
  • Do not invoke single file validators if table indices were not generated by @aababilov in #1300
  • Always invoke single file validators (revert #1300) by @aababilov in #1303
  • Discover tables and validators in runtime using io.github.classgraph by @aababilov in #1311
  • Initialize one field per line in the generated code by @aababilov in #1314
  • feat: Refactor JSON report comparison for better diffing in unit-tests. by @bdferris-v2 in #1352
  • feat: Move key annotation and entity interfaces into a new model module by @bdferris-v2 in #1362
  • feat: Initial support for GTFS Fares v2 - fare media by @bdferris-v2 in #1305
  • feat: Add a more descriptive test failure message to NoticeFieldsTest. by @bdferris-v2 in #1369
  • chore: Add unit tests to AnyTableLoader by @davidgamez in #1380

New Contributors

Full Changelog: v4.0.0...v4.1.0

Contributors

briandonahue, KClough, and 7 other contributors
Assets 6
Loading

v4.0.0

25 Oct 01:55
@isabelle-dr isabelle-dr
v4.0.0
35bf825
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v4.0.0

Overview

This is tagged as a major release because adding the rules for Fares v2 base implementation is affecting the validity of existing datasets. We started to evaluate the impact of new rules on production data in 2022 with the acceptance tests, which runs this validator on all datasets from the Mobility Database, and reports if additional errors appear. When new errors appear, we publish a major release.

New notices

GTFS-Fares v2 base implementation, added in google/transit/pull/286

  • fare_transfer_rule_duration_limit_type_without_duration_limit 🔴 ERROR
  • fare_transfer_rule_duration_limit_without_type 🔴 ERROR
  • fare_transfer_rule_invalid_transfer_count 🔴 ERROR
  • fare_transfer_rule_missing_transfer_count 🔴 ERROR
  • fare_transfer_rule_with_forbidden_transfer_count 🔴 ERROR
  • invalid_currency_amount 🔴 ERROR

Transfers, changes in google/transit/pull/303 and google/transit/pull/284

  • transfer_with_invalid_stop_location_type 🔴 ERROR
  • transfer_with_invalid_trip_and_route 🔴 ERROR
  • transfer_with_invalid_trip_and_stop 🔴 ERROR
  • transfer_with_suspicious_mid_trip_in_seat 🟠 WARNING

Best Practice rules

  • missing_recommended_file (applies to feed_info.txt) 🟠 WARNING
  • missing_recommended_field (applies to feed_info.txt start_date, end_date, email, URL, version) 🟠 WARNING)
  • feed_expiration_date_7_days (replacing feed_expiration_date) 🟠 WARNING
  • feed_expiration_date_30_days (replacing feed_expiration_date) 🟠 WARNING

Impacts on existing data

On all datasets MobilityData ran the analytics on, we found that 73 sources would be affected by this release. The datasets affected mostly contain earlier versions of Fares v2 that don't match what was adopted in the spec, and uncertain cases, future versions.
The detailed analysis is in the Fares v2 base implementation for the validator document.

Producers affected

  • Trillium Transit
  • San Diego Metropolitan Transit System

Complete list

See the complete list of datasets affected 1/2
URL Notices
http://data.trilliumtransit.com/gtfs/nts-ca-us/nts-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/victorville-ca-us/victorville-ca-us.zip duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/tracy-ca-us/tracy-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/downey-ca-us/downey-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/weaverville-ca-us/weaverville-ca-us.zip duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/laketransit-ca-us/laketransit-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/auburntransit-ca-us/auburntransit-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/cityofridgecrest-ca-us/cityofridgecrest-ca-us.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
https://www.sdmts.com/google_transit_files/google_transit.zip duplicate_key,fare_transfer_rule_missing_transfer_count,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/lagunabeach-ca-us/lagunabeach-ca-us.zip invalid_currency_amount
http://data.trilliumtransit.com/gtfs/goldcountrystage-ca-us/goldcountrystage-ca-us.zip duplicate_key,fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
https://transitfeeds.com/p/santa-cruz-metro/343/latest/download duplicate_key,invalid_currency_amount,missing_required_field
http://data.trilliumtransit.com/gtfs/redding-ca-us/redding-ca-us.zip duplicate_key,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/corona-ca-us/corona-ca-us.zip fare_transfer_rule_missing_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/gtrans-ca-us/gtrans-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/vctc-ca-us/vctc-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sanbenitocounty-ca-us/sanbenitocounty-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/delnorte-ca-us/delnorte-ca-us.zip fare_transfer_rule_with_forbidden_transfer_count,foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/plumas-ca-us/plumas-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/sagestage-ca-us/sagestage-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/bigbear-ca-us/bigbear-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/mercedthebus-ca-us/mercedthebus-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/thousandoaks-ca-us/thousandoaks-ca-us.zip foreign_key_violation,invalid_currency_amount
See the complete list of datasets affected 2/2
URL Notices
http://data.trilliumtransit.com/gtfs/lassen-ca-us/lassen-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/banning-ca-us/banning-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/morongobasin-ca-us/morongobasin-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/amador-ca-us/amador-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/gtfs/glenn-ca-us/glenn-ca-us.zip foreign_key_violation,invalid_currency_amount
http://data.trilliumtransit.com/...
Read more

Contributors

KClough, themightychris, and 6 other contributors
Assets 8
Loading
2 Join discussion

v3.1.1

28 Jun 23:21
@isabelle-dr isabelle-dr
v3.1.1
ceb2a86
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.1.1

Bug fixes

  • fix: Bug with ssl network connections + Java module permissions. by @bdferris-v2 in #1182
  • fix: Temporary fix for MacOS validator app corruption by @bdferris-v2 in #1186

Enhancements

New Contributors

Full Changelog: v3.1.0...v3.1.1

Contributors

KClough and bdferris-v2
Assets 10
Loading

v3.1.0

03 Jun 21:06
@isabelle-dr isabelle-dr
v3.1.0
da57dbf
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.1.0

📣 New feature alert! Users can now run the validator as a packaged app and see the results in the browser. This work is described in the epic issue #1124

Enhancements

Bugs fixed

Documentation improvements

Maintenance and upkeep improvements

Contributors to this release

Full Changelog: v3.0.1...v3.1.0

Contributors

barbeau, ed-g, and 4 other contributors
Assets 8
Loading

v3.0.1

16 Mar 20:21
@isabelle-dr isabelle-dr
v3.0.1
b8debd9
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.0.1

Main update

The main difference between the v3.0.0 and v3.0.1 is the modification of ShapeIncreasingDistanceValidator. After a discussion in #1070, we realized that we had a problem with the notice DecreasingOrEqualShapeDistanceNotice (previously an ERROR): because of the way scheduling software produce data, we can have records in shapes.txt that have equal coordinates, and equal values for shape_dist_traveled. This can happen when two shape points are really close to one another. It creates a somewhat duplicative record, and this should not be an ERROR. A true ERROR happens if values of shape_distance_traveled are equal for two shape points that have different coordinates.

We decided to replace this notice with three new ones (in PR #1083):
1- DecreasingShapeDistanceNotice as an ERROR: if two consecutive shape points have decreasing values for shape_dist_traveled.
2- EqualShapeDistanceDiffCoordinatesNotice as an ERROR: if two consecutive points have equal shape_dist_traveled and different lat/lon coordinates in shapes.txt.
3- EqualShapeDistanceSameCoordinatesNotice as a WARNING: if two record have the same values for shape_dist_traveled and lat/lon coordinates in shapes.txt.

List of merged PRs

New Contributors

Full Changelog: v3.0.0...v3.0.1

Contributors

aababilov, barbeau, and 3 other contributors
Assets 5
Loading

v3.0.0

10 Jan 16:13
@isabelle-dr isabelle-dr
v3.0.0
fed4bad
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v3.0.0

We're excited to bring you the 3.0.0 release, it's a big one!

We three had main goals for this release:

➡️ Solve discrepancies with the official GTFS specification.
26 new rules have been added in total:

  • New validation rules were added for the adopted GTFS extensions pathways.txt, levels.txt, translations.txt
  • New custom rules were added for parts of the spec that were missing from this validator
  • Some existing rules were improved to better reflect the specification
  • The severity of some rules was updated to reflect the specification after the update to RFC 2119 has been done

Please refer to the V2.0.0 to V3.0.0 rule mapping available in NOTICE_MIGRATION.md for more details on the rule updates.

➡️ Implement automated tests that will evaluate if a new rule results in datasets from the mobility database getting additional errors. If so, the Pull Request is blocked and a report will be generated for further investigation. This ensures the stability and reliability of this validator, while still being updated with updates and extensions in the specification.

➡️ Make sure MobilityData's work has related public communication to facilitate community contribution (issues opened, public sprint board and bug tracking board, roadmap, ways for users to voice what they'd like to see next and for contributors to see how to get involved).

There are also new command line parameters that can be found in USAGE.md.

There is a lot more work included in this release, thanks to our amazing contributors. See details below!

Work ⚙️

Solve discrepancies with the spec

  • chore: downgrade LeadingOrTrailingWhitespacesNotice to WARNING in #929
  • chore: upgrade EmptyColumnNameNotice to ERROR in #1019
  • chore: new rule - additional requirements urls (stops.txt, routes.txt, agency.txt) in #932

routes.txt

  • feat: Reduce severity of SameNameAndDescriptionForRouteNotice to WARNING in #917

stops.txt

  • chore: check stop name and desc in #937
  • chore: check stops.zone_id presence in #936

stop_times.txt

  • chore: new rule - additional check on stops.location_type in #938
  • chore: upgrade LocationWithUnexpectedStopTimeNotice to ERROR in #1021

pathways.txt and levels.txt

  • feat: Validate how locations are referenced from stop_times.stop_id in #960
  • feat: Validate that pathway endpoints have correct types in #961
  • feat: verify conditional requirement of levels.txt presence in #966
  • feat: additional checks on pathways.txt and stops.txt in #967
  • feat: Validate that locations are reachable if a station has pathways in #997
  • feat: Validate that a pathway is not a loop in #999
  • feat: Validate field presence and key integrity for translations.txt in #968
  • chore: upgrade PathwayUnreachableLocationNotice to ERROR in #1028

translations.txt

  • chore: upgrade TranslationForeignKeyViolationNotice to ERROR in #1027

Automated test to see if a PR results in additional errors

Other work included

Better notice schema export

Better documentation

Bugs

  • fix: Prevent NullPointerException when Throwable.getMessage returns null in #871
  • fix: Catch any runtime exception when parsing a GTFS field in #1012
  • fix: Swap arguments of isAssignableFrom in #872
  • fix: bump debian version used in docker image in #891
  • fix: Protect against start_date and end_date in wrong order in #897
  • fix: Raise IllegalArgumentException for invalid date in #977
  • fix: change limit on notice count during validation to 100_000 in #1018
  • fix: validate route, stops and agency consistency urls in #1003
  • feat: Prevent OOM in NoticeContainer and speed up hasValidationErrors in #895
  • fix: Skip phone number validation when country code is unknown in #1062
  • fix: False positives for StopTimeTimepointWithoutTimesNotice in #1044

Other good stuff

  • ci: check code formatting on pull requests in #953
  • ci: publish a new docker image on every push on master in #913
  • ci: verify gradle wrapper in #964
  • ci: bug triage in #982
  • feat: Provide enum setters and clearFIELD methods in #843
  • feat: make report names user configurable via CLI args in #852
  • feat!: deprecate -f in #851
  • feat: Pass severityLevel to constructor of some notices in #863
  • feat: Propagate InterruptedException during feed loading in #869
  • feat: Catch CSV parse errors and add validation notices in #874
  • feat: print help menu in #888
  • feat: Export infinity and NaN values in JSON in #889
  • feat: Treat empty files as parsed unsuccessfully in #898
  • feat: introduce -p CLI parameter to output beautified reports in #901
  • feat: Add byPrimaryKey and getKeyColumnNames methods to GtfsTableContainer in #1000
  • chore: upgrade to gradle wrapper to 7.2 in #965
  • chore: Upgrade to Java 11 in #1014

Contributors 🙏

Massive shoutout to:
🎉 @aababilov
🎉 @asvechnikov2
🎉 @barbeau
🎉 @lionel-nj
🎉 @maximearmstrong
🎉 @isabelle-dr

Contributors

aababilov, barbeau, and 4 other contributors
Assets 5
Loading