Skip to content

Conversation

@j-so
Copy link
Contributor

@j-soj-so commented May 21, 2020

Introduces a new set of pipelines that split model/train/eval and deployment.

  1. Model training/eval/registration - performs Model CI, trains, evaluates, and registers the model. If the model is registered, then it produces a pipeline artifact containing the model JSON.
  2. Model deployment - takes the pipeline artifact (model JSON) from the triggering pipeline and deploys that model.

NOTE: There is a limitation with the model deployment pipeline. It relies solely on the output of the model training pipeline. Right now it cannot run off the model name/version env vars.

Also addresses #274

@j-soj-so marked this pull request as ready for review May 27, 2020 00:40
@j-soj-so linked an issue Jun 9, 2020 that may be closed by this pull request
j-so added 4 commits June 10, 2020 12:19
commit 01af8da Author: j-so <[email protected]> Date: Fri Jun 12 14:15:35 2020 -0700 fixed failure handling commit 1e6f906 Author: j-so <[email protected]> Date: Fri Jun 12 14:03:09 2020 -0700 test failed conda dep commit a8030d7 Author: j-so <[email protected]> Date: Fri Jun 12 13:55:00 2020 -0700 test package fail commit c7845aa Author: j-so <[email protected]> Date: Fri Jun 12 13:46:27 2020 -0700 fail on deploy error'
Copy link
Contributor

@dtzardtzar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the CI steps are identical, I believe we should keep thing consistent by having a CI and then a CD pipeline which is triggered when the CI pipeline is complete. Sooo... This means for this PR:

  1. No need for a separate "getting started doc"
  2. Remove duplicate CI pipeline, fix the existing one to move the CD steps into a "Task based deployment" pipeline.
  3. Fix the main getting started guide to ask people if they want the AzDO task deployment or Az CLI deployment, setting whatever you feel is the best default one triggered by the CI pipeline. If you feel the engineering team doesn't want people to have the task deployment, then go ahead and remove this choice.

j-so added 2 commits June 15, 2020 18:32
commit 5af6eeb Author: j-so <[email protected]> Date: Mon Jun 15 18:31:15 2020 -0700 fix bootstrap commit f61e103 Merge: 2796b4008bb6f4 Author: j-so <[email protected]> Date: Mon Jun 15 18:30:21 2020 -0700 Merge branch 'master' into jenns/splitpipeline_docfix commit 2796b40 Author: j-so <[email protected]> Date: Mon Jun 15 18:30:00 2020 -0700 remove old pipeline commit 08bb6f4 Author: David Tesar <[email protected]> Date: Mon Jun 15 14:09:12 2020 -0700 Simplify docs flow (#297) commit cd762ec Author: jotaylo <[email protected]> Date: Mon Jun 15 12:28:23 2020 -0700 Move instruction to install AML extension to Azure Devops setup instructions (#298)
j-so added 6 commits June 18, 2020 14:38
commit 493ed3e Author: j-so <[email protected]> Date: Mon Jun 22 16:42:07 2020 -0700 mark as output commit 1ca7a59 Author: j-so <[email protected]> Date: Mon Jun 22 16:12:10 2020 -0700 fix import commit 743e301 Author: j-so <[email protected]> Date: Mon Jun 22 15:59:43 2020 -0700 more fixes commit 44abcac Author: j-so <[email protected]> Date: Mon Jun 22 15:50:49 2020 -0700 fix batch scoring
@j-soj-so requested a review from jitghoshJune 23, 2020 03:42
@j-soj-so changed the title Adding new split pipeline approach for CI/CDConvert to new split pipeline approach for CI/CDJun 24, 2020
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Q: Run pipeline on non-model changes

4 participants

@j-so@dtzar@daniel-hanson