Adding ArangoDB Provider#22548
Merged
Adding ArangoDB Provider #22548
Uh oh!
There was an error while loading. Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
eladkal reviewed Mar 27, 2022
Uh oh!
There was an error while loading. Please reload this page.
| :param arangodb_conn_id: Reference to :ref:`ArangoDB connection id <howto/connection:arangodb>`. | ||
| """ | ||
| template_fields: Sequence[str] = ('sql',) |
Contributor
There was a problem hiding this comment.
Should we add also template_ext?
| :param arangodb_db: Target ArangoDB name. | ||
| """ | ||
| template_fields: Sequence[str] = ('sql',) |
Contributor
There was a problem hiding this comment.
Should we add also template_ext?
Uh oh!
There was an error while loading. Please reload this page.
Member
potiuk commented Mar 27, 2022
Can you fix static checks please. |
ContributorAuthor
pateash commented Mar 29, 2022
|
ContributorAuthor
pateash commented Mar 29, 2022
|
Contributor
eladkal commented Mar 30, 2022
@potiuk can you take a look at the test failure? I don't recall that when adding a new provider we need to edit the CI script |
Member
Not everything in providers has to be me :) - this test was added by @mik-laj actually: 621d17b It looks like for some reason prodcution image produced in this build contains many more providers than it should |
Member
potiuk commented Mar 30, 2022
Yeah: seems that for some reason it contains all providers: |
Member
potiuk commented Mar 30, 2022
This is VERY strange as it seems that when the image was built, it actually used only a small subset (as expected): |
Member
potiuk commented Mar 30, 2022
Let me rebase and see it happening again :) |
Contributor
eladkal commented Mar 30, 2022
I don't recall we had such issue when GitHub provider was added (and it was after 621d17b ) |
Member
potiuk commented Mar 30, 2022
Me neither. It basicallly SHOUD NOT happen :D. Yet it seems it did again |
Member
potiuk commented Mar 30, 2022
OK. I know what causes it but I do not know why it happens yet. When PROD build image is prepared we prepare "airflow" package so that it can be installed there from latest sources. But for SOME reason, it contains "all" providers as well. not only airflow. I do not know where it came from yet. But It proves the tests from @mik-laj are useful to catch it. |
Member
potiuk commented Mar 30, 2022
I actually think it could come from the new setuptools release https://pypi.org/project/setuptools/61.2.0/ |
Member
potiuk commented Mar 30, 2022
Still puzzled :) but I am getting closer to solve it |
ContributorAuthor
pateash commented Mar 30, 2022
thanks @potiuk. |
Member
potiuk commented Mar 31, 2022
Member
potiuk commented Apr 1, 2022
ContributorAuthor
voila 🥳, |
potiuk approved these changes Apr 1, 2022
github-actionsbot added the full tests needed 
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease. |
Contributor
eladkal commented Apr 1, 2022
I'll take a look later today |
eladkal approved these changes Apr 3, 2022
| +---------------------+-----------------------------------------------------+-------------------------------------------+ | ||
| | trino | ``pip install 'apache-airflow[trino]'`` | All Trino related operators & hooks | | ||
| +---------------------+-----------------------------------------------------+-------------------------------------------+ | ||
| | arangodb | ``pip install 'apache-airflow[arangodb]'`` | ArangoDB operators, sensors and hook | |
Contributor
There was a problem hiding this comment.
I think this list is sorted alphabetically?
Member
potiuk commented Apr 3, 2022
🎉 🎉 🎉 🎉 🎉 🎉 🎉 |
ephraimbuddy added the changelog:skip Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes: #17778
Description
Adding ArangoDB provider based on Python SDK https://github.com/ArangoDB-Community/python-arango
Users can create their own custom operators leveraging the ArangoDBHook directly
or building their operator on AQLOperator by providing result_processor method,
Sensor can be implemented by SQL