Skip to content

README.md (#181) #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,6 @@ op

# Coverage
coverage.txt

# Misc
.DS_Store
21 changes: 13 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,10 +9,15 @@ Omniparser is a native Golang ETL parser that ingests input data of various form
XML, EDI/X12/EDIFACT, JSON**, and custom formats) in streaming fashion and transforms data into desired JSON output
based on a schema written in JSON.

**Please kindly consider sponsoring the project to fund future development and issue resolutions**: https://github.com/sponsors/jf-tech

Min Golang Version: 1.14

## Licenses and Sponsorship
Omniparser is publicly available under [MIT License](./LICENSE).
[Individual and corporate sponsorships](https://github.com/sponsors/jf-tech/) are welcome and gratefully
appreciated, and will be listed in the [SPONSORS](./sponsors/SPONSORS.md) page.
[Company-level sponsors](https://github.com/sponsors/jf-tech/) get additional benefits and supports
granted in the [COMPANY LICENSE](./sponsors/COMPANY_LICENSE.md).

## Documentation

Docs:
Expand Down Expand Up @@ -46,12 +51,12 @@ Examples:
In the example folders above you will find pairs of input files and their schema files. Then in the
`.snapshots` sub directory, you'll find their corresponding output files.

## Online Playground
## Online Playground (not functioning)

Use https://omniparser.herokuapp.com/ (may need to wait for a few seconds for heroku instance to wake up)
for trying out schemas and inputs, yours or existing samples, to see how ingestion and transform work.
~~Use [The Playground](https://omniparser-prod-omniparser-qd0sj4.mo2.mogenius.io/) (may need to wait for a few seconds for instance to wake up)
for trying out schemas and inputs, yours or existing samples, to see how ingestion and transform work.~~

![](./cli/cmd/web/playground-demo.gif)
As for now (2023/03/14), all of our previous free docker hosting solutions went away and we haven't found another one yet. For now please clone the repo and use `./cli.sh` as described in the [Getting Started](./doc/gettingstarted.md) page.

## Why
- No good ETL transform/parser library exists in Golang.
Expand Down Expand Up @@ -83,8 +88,8 @@ back-compatibility, it is just removed from all public docs and samples).
- `'keep_leading_trailing_space'` -> `'no_trim'`
- Changed how we handle custom functions: previously we always use strings as in param type as well as result param
type. Not anymore, all types are supported for custom function in and out params.
- Changed the way how we package custom functions for extensions: previously we collect custom functions from all
extensions and then pass all of them to the extension that is used; This feels weird, now changed to only the custom
- Changed the way we package custom functions for extensions: previously we collected custom functions from all
extensions and then passed all of them to the extension that is used; this feels weird, now only the custom
functions included in a particular extension are used in that extension.
- Deprecated/removed most of the custom functions in favor of using 'javascript'.
- A number of package renaming.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"child_segments": [
{
"name": "GS",
"_comment": "functional group",
"child_segments": [
{
"name": "scanInfo", "type": "segment_group", "min": 0, "max": -1, "is_target": true,
Expand All @@ -26,7 +27,7 @@
{
"name": "N4",
"elements": [
{ "name": "cityName", "index": 1 },
{ "name": "cityName", "index": 1, "_comment": "E19" },
{ "name": "provinceCode", "index": 2 },
{ "name": "postalCode", "index": 3 },
{ "name": "countryCode", "index": 4 }
Expand Down
3 changes: 2 additions & 1 deletion extensions/omniv21/samples/edi/2_ups_edi_210.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
"child_segments": [
{
"name": "GS",
"_comment": "functional group",
"child_segments": [
{
"name": "invoiceInfo", "type": "segment_group", "min": 0, "max": -1, "is_target": true,
Expand Down Expand Up @@ -74,7 +75,7 @@
"elements": [
{ "name": "refIdQualifier_N901", "index": 1 },
{ "name": "refId_N902", "index": 2, "default": "" },
{ "name": "date", "index": 4, "default": "" },
{ "name": "date", "index": 4, "default": "", "_comment": "YYYYMMDD"},
{ "name": "product_code_C04004", "index": 7, "component_index": 4, "default": "" }
]
},
Expand Down
9 changes: 6 additions & 3 deletions extensions/omniv21/validation/ediFileDeclaration.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 6 additions & 3 deletions extensions/omniv21/validation/ediFileDeclaration.json
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@
"index": { "type": "integer", "minimum": 1 },
"component_index": { "type": "integer", "minimum": 1 },
"empty_if_missing": { "type": "boolean","$comment": "deprecated, use 'default'" },
"default": { "type": "string" }
"default": { "type": "string" },
"_comment": { "$ref": "#/definitions/value_comment" }
},
"required": [ "name", "index" ],
"additionalProperties": false
Expand All @@ -53,10 +54,12 @@
"items": {
"$ref": "#/definitions/segment_declaration_type"
}
}
},
"_comment": { "$ref": "#/definitions/value_comment" }
},
"required": [ "name" ],
"additionalProperties": false
}
},
"value_comment": { "type": "string" }
}
}
23 changes: 23 additions & 0 deletions sponsors/COMPANY_LICENSE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Company License

Copyright (c) 2023 - present [JF Technology](https://github.com/jf-tech)

## Licensing Information

In addition to the terms and conditions mentioned in the [main license](../LICENSE), your
company is entitled to the following when company level sponsorship is active:

| Item | - |
| ---------------------------------- | --- |
| Team Size | **Unlimited** |
| Number of Apps/Services Using This Code | **Unlimited** |
| Feature Updates and Bug Fixes: | **Yes** |
| Prioritized Feature Request Discussion and Response | **Yes** |
| Prioritized [Support](mailto:[email protected]) or [Issues](https://github.com/jf-tech/omniparser/issues) Response | **Yes** |
| Included in [Sponsors Page](./SPONSORS.md) | **Yes** |
| --- | --- |
| Remove copyright mentions/license from source code | NO |
| Production deployment and troubleshooting assistance | NO |

---
For more information regarding licensing, please contact [support](mailto:[email protected]).
11 changes: 11 additions & 0 deletions sponsors/SPONSORS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Sponsors & Backers

> Omniparser is an MIT-licensed open source project with its ongoing development made possible entirely by the support of the awesome sponsors and backers listed in this file. If you'd like to join them, please consider [ sponsor its development](https://github.com/sponsors/jf-tech).

### Corporate Sponsors

- Entos LLC

- ![HealthSherpa](resources/HealthSherpa.png)

### Generous Individual Backers
Binary file added sponsors/resources/HealthSherpa.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added sponsors/resources/template.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.