- Notifications
You must be signed in to change notification settings - Fork 1.1k
Fix/sqlalchemy 2.0 compatibility - chapter6 UoW#110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
rickywesker wants to merge 67 commits into cosmicpython:chapter_05_uow_exerciseChoose a base branch from rickywesker:fix/sqlalchemy-2.0-compatibility
base:chapter_05_uow_exercise
Could not load branches
Branch not found: {{refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline, and old review comments may become outdated.
Uh oh!
There was an error while loading. Please reload this page.
Open
Changes from all commits
Commits
Show all changes
67 commits Select commit Hold shift + click to select a range
9a74b16 makefile for running stuff
hjwp 5be7bcf first test [first_test]
hjwp fc5811e first stab at a model [domain_model_1]
hjwp 108a74c more tests for can_allocate [test_can_allocate]
hjwp d817ecd can_allocate fn [can_allocate]
hjwp bd0e75e simple deallocate test
hjwp 8ad3236 simple deallocate function
hjwp bebd0fd test deallocate not allocated [test_deallocate_unallocated]
hjwp b2d618a model now tracks allocations [domain_model_complete]
hjwp ad26a08 test allocate twice [last_test]
hjwp 235e121 equality and hash operators [equality_on_batches]
hjwp d1dda70 new tests for allocate domain service [test_allocate]
hjwp 8a6ec0e allocate fn, domain service [domain_service]
hjwp ff0a15c fixup a batchref
hjwp 0c3b87e change tests add one for return
hjwp 07dd382 make Batches sortable [dunder_gt]
hjwp 18aa3b4 fixup a sku
hjwp d75eaf7 test out of stock exception [test_out_of_stock]
hjwp 184d90d raising out of stock exception [out_of_stock]
hjwp 73dae3f add readme from master
hjwp 5b77fc4 Wrong path in venv creation line
karolpawlowski 3e9871d travis config. [chapter_01_domain_model_ends]
hjwp 58d05e4 first cut of orm, orderlines only [sqlalchemy_classical_mapper]
hjwp ef7a621 first tests of orm [orm_tests]
hjwp 527b320 unfortunate hack on dataclass in model
hjwp 26b997e batches with no allocations
hjwp fff68bd ORM for _allocations set on Batch
hjwp 8dae126 repository tests
hjwp b635a15 repository for batches [chapter_02_repository_ends]
hjwp fda077d first api tests [first_api_test]
hjwp 15902a6 all the dockerfile gubbins
hjwp 8fc086e first cut of flask app [first_cut_flask_app]
hjwp 158e760 test persistence by double-allocating. [second_api_test]
hjwp d0ee7ad need to commit [flask_commit]
hjwp f20c479 test some 400 error cases [test_error_cases]
hjwp 80dece9 flask now does error handling [flask_error_handling]
hjwp a1903fa first tests for the services layer [first_services_tests]
hjwp 9fe9a3d FakeRepository [fake_repo]
hjwp 15b9bf7 FakeSession [fake_session]
hjwp eded7eb test commmits [second_services_test]
hjwp d1e2e6e services layer with valid-sku check [service_function]
hjwp cf2f52b modify flask app to use service layer [flask_app_using_service_layer]
hjwp b537364 strip out unecessary tests from e2e layer [fewer_e2e_tests]
hjwp 952a3d2 fix conftest waits and travis config [chapter_04_service_layer_ends]
hjwp bdf8fe9 move to a more nested folder structure
hjwp 1bb572a nest the tests too
hjwp db89218 get all tests passing
hjwp c5822aa rewrite service layer to take primitives [service_takes_primitives]
hjwp f341bee services tests partially converted to primitives [tests_call_with_pri…
hjwp 1e1f238 fixture function for batches [services_factory_function]
hjwp 5c953da new service to add a batch [add_batch_service]
hjwp 262eec0 service-layer test for add batch [test_add_batch]
hjwp c8fbb60 all service-layer tests now services [services_tests_all_services]
hjwp 96301d2 modify flask app to use new service layer api [api_uses_modified_serv…
hjwp 6b404b5 add api endpoint for add_batch [api_for_add_batch]
hjwp fd45a6f api tests no longer need hardcoded sql fixture [chapter_05_high_gear_…
hjwp d9c340c start moving files into src folder and add setup.py
hjwp c843a10 fix all the imports, get it all working
hjwp 12d2bc2 get tests working in docker container
hjwp 28afa9a make mypy slightly stricter
hjwp 53ad798 better requirements.txt [appendix_project_structure_ends]
hjwp 833d48b basic uow test, uow and conftest.py changes
hjwp 658e61a use uow in services, flask app
hjwp 7526014 two more tests for rollback behaviour [chapter_06_uow_ends]
hjwp 916bcdd strip out UoW and fake UoW, add some tips
hjwp 6876e75 update tests
hjwp 00f1574 fix: SQLAlchemy 2.0 compatibility for chapter_06_uow_exercise
rickywesker File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading
Uh oh!
There was an error while loading. Please reload this page.
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| dist: xenial | ||
| language: python | ||
| python: 3.8 | ||
| script: | ||
| - make all | ||
| branches: | ||
| except: | ||
| - /.*_exercise$/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| FROM python:3.9-slim-buster | ||
| # RUN apt install gcc libpq (no longer needed bc we use psycopg2-binary) | ||
| COPY requirements.txt /tmp/ | ||
| RUN pip install -r /tmp/requirements.txt | ||
| RUN mkdir -p /src | ||
| COPY src/ /src/ | ||
| RUN pip install -e /src | ||
| COPY tests/ /tests/ | ||
| WORKDIR /src | ||
| ENV FLASK_APP=allocation/entrypoints/flask_app.py FLASK_DEBUG=1 PYTHONUNBUFFERED=1 | ||
| CMD flask run --host=0.0.0.0 --port=80 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| # these will speed up builds, for docker-compose >= 1.25 | ||
| export COMPOSE_DOCKER_CLI_BUILD=1 | ||
| export DOCKER_BUILDKIT=1 | ||
| all: down build up test | ||
| build: | ||
| docker-compose build | ||
| up: | ||
| docker-compose up -d app | ||
| down: | ||
| docker-compose down --remove-orphans | ||
| test: up | ||
| docker-compose run --rm --no-deps --entrypoint=pytest app /tests/unit /tests/integration /tests/e2e | ||
| unit-tests: | ||
| docker-compose run --rm --no-deps --entrypoint=pytest app /tests/unit | ||
| integration-tests: up | ||
| docker-compose run --rm --no-deps --entrypoint=pytest app /tests/integration | ||
| e2e-tests: up | ||
| docker-compose run --rm --no-deps --entrypoint=pytest app /tests/e2e | ||
| logs: | ||
| docker-compose logs app | tail -100 | ||
| black: | ||
| black -l 86 $$(find * -name '*.py') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Example application code for the python architecture book | ||
| ## Chapters | ||
| Each chapter has its own branch which contains all the commits for that chapter, | ||
| so it has the state that corresponds to the _end_ of that chapter. If you want | ||
| to try and code along with a chapter, you'll want to check out the branch for the | ||
| previous chapter. | ||
| https://github.com/python-leap/code/branches/all | ||
| ## Exercises | ||
| Branches for the exercises follow the convention `{chatper_name}_exercise`, eg | ||
| https://github.com/python-leap/code/tree/chapter_04_service_layer_exercise | ||
| ## Requirements | ||
| * docker with docker-compose | ||
| * for chapters 1 and 2, and optionally for the rest: a local python3.7 virtualenv | ||
| ## Building the containers | ||
| _(this is only required from chapter 3 onwards)_ | ||
| ```sh | ||
| make build | ||
| make up | ||
| # or | ||
| make all # builds, brings containers up, runs tests | ||
| ``` | ||
| ## Creating a local virtualenv (optional) | ||
| ```sh | ||
| python3.8 -m venv .venv && source .venv/bin/activate # or however you like to create virtualenvs | ||
| # for chapter 1 | ||
| pip install pytest | ||
| # for chapter 2 | ||
| pip install pytest sqlalchemy | ||
| # for chapter 4+5 | ||
| pip install requirements.txt | ||
| # for chapter 6+ | ||
| pip install requirements.txt | ||
| pip install -e src/ | ||
| ``` | ||
| <!-- TODO: use a make pipinstall command --> | ||
| ## Running the tests | ||
| ```sh | ||
| make test | ||
| # or, to run individual test types | ||
| make unit | ||
| make integration | ||
| make e2e | ||
| # or, if you have a local virtualenv | ||
| make up | ||
| pytest tests/unit | ||
| pytest tests/integration | ||
| pytest tests/e2e | ||
| ``` | ||
| ## Makefile | ||
| There are more useful commands in the makefile, have a look and try them out. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| version: "3" | ||
| services: | ||
| app: | ||
| build: | ||
| context: . | ||
| dockerfile: Dockerfile | ||
| depends_on: | ||
| - postgres | ||
| environment: | ||
| - DB_HOST=postgres | ||
| - DB_PASSWORD=abc123 | ||
| - API_HOST=app | ||
| - PYTHONDONTWRITEBYTECODE=1 | ||
| volumes: | ||
| - ./src:/src | ||
| - ./tests:/tests | ||
| ports: | ||
| - "5005:80" | ||
| postgres: | ||
| image: postgres:9.6 | ||
| environment: | ||
| - POSTGRES_USER=allocation | ||
| - POSTGRES_PASSWORD=abc123 | ||
| ports: | ||
| - "54321:5432" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,7 @@ | ||
| [mypy] | ||
| ignore_missing_imports = False | ||
| mypy_path = ./src | ||
| check_untyped_defs = True | ||
| [mypy-pytest.*] | ||
| [mypy-pytest.*,sqlalchemy.*] | ||
| ignore_missing_imports = True | ||
| [mypy-sqlalchemy.*] | ||
| ignore_missing_imports = True | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # app | ||
| sqlalchemy>=2.0 | ||
| flask | ||
| psycopg2-binary | ||
| # tests | ||
| pytest | ||
| pytest-icdiff | ||
| mypy | ||
| requests |
Empty file.
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,50 @@ | ||
| from sqlalchemy import Table, MetaData, Column, Integer, String, Date, ForeignKey | ||
rickywesker marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading. Please reload this page. | ||
| from sqlalchemy.orm import registry, relationship | ||
| from allocation.domain import model | ||
| mapper_registry = registry() | ||
| metadata = mapper_registry.metadata | ||
| order_lines = Table( | ||
| "order_lines", | ||
| metadata, | ||
| Column("id", Integer, primary_key=True, autoincrement=True), | ||
| Column("sku", String(255)), | ||
| Column("qty", Integer, nullable=False), | ||
| Column("orderid", String(255)), | ||
| ) | ||
| batches = Table( | ||
| "batches", | ||
| metadata, | ||
| Column("id", Integer, primary_key=True, autoincrement=True), | ||
| Column("reference", String(255)), | ||
| Column("sku", String(255)), | ||
| Column("_purchased_quantity", Integer, nullable=False), | ||
| Column("eta", Date, nullable=True), | ||
| ) | ||
| allocations = Table( | ||
| "allocations", | ||
| metadata, | ||
| Column("id", Integer, primary_key=True, autoincrement=True), | ||
| Column("orderline_id", ForeignKey("order_lines.id")), | ||
| Column("batch_id", ForeignKey("batches.id")), | ||
| ) | ||
| def start_mappers(): | ||
| lines_mapper = mapper_registry.map_imperatively(model.OrderLine, order_lines) | ||
| mapper_registry.map_imperatively( | ||
| model.Batch, | ||
| batches, | ||
| properties={ | ||
| "_allocations": relationship( | ||
| lines_mapper, | ||
| secondary=allocations, | ||
| collection_class=set, | ||
| ) | ||
| }, | ||
| ) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| import abc | ||
| from sqlalchemy import select | ||
| from allocation.domain import model | ||
| class AbstractRepository(abc.ABC): | ||
| @abc.abstractmethod | ||
| def add(self, batch: model.Batch): | ||
| raise NotImplementedError | ||
| @abc.abstractmethod | ||
| def get(self, reference) -> model.Batch: | ||
| raise NotImplementedError | ||
| class SqlAlchemyRepository(AbstractRepository): | ||
| def __init__(self, session): | ||
| self.session = session | ||
| def add(self, batch): | ||
| self.session.add(batch) | ||
| def get(self, reference): | ||
| return self.session.scalars( | ||
| select(model.Batch).filter_by(reference=reference) | ||
| ).one() | ||
| def list(self): | ||
| return self.session.scalars(select(model.Batch)).all() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| import os | ||
| def get_postgres_uri(): | ||
| host = os.environ.get("DB_HOST", "localhost") | ||
| port = 54321 if host == "localhost" else 5432 | ||
| password = os.environ.get("DB_PASSWORD", "abc123") | ||
| user, db_name = "allocation", "allocation" | ||
| return f"postgresql://{user}:{password}@{host}:{port}/{db_name}" | ||
| def get_api_url(): | ||
| host = os.environ.get("API_HOST", "localhost") | ||
| port = 5005 if host == "localhost" else 80 | ||
| return f"http://{host}:{port}" |
Empty file.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| from __future__ import annotations | ||
| from dataclasses import dataclass | ||
| from datetime import date | ||
| from typing import Optional, List, Set | ||
| class OutOfStock(Exception): | ||
| pass | ||
| def allocate(line: OrderLine, batches: List[Batch]) -> str: | ||
| try: | ||
| batch = next(b for b in sorted(batches) if b.can_allocate(line)) | ||
| batch.allocate(line) | ||
| return batch.reference | ||
| except StopIteration: | ||
| raise OutOfStock(f"Out of stock for sku{line.sku}") | ||
| @dataclass(unsafe_hash=True) | ||
| class OrderLine: | ||
| orderid: str | ||
| sku: str | ||
| qty: int | ||
| class Batch: | ||
rickywesker marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading. Please reload this page. | ||
| def __init__(self, ref: str, sku: str, qty: int, eta: Optional[date]): | ||
| self.reference = ref | ||
| self.sku = sku | ||
| self.eta = eta | ||
| self._purchased_quantity = qty | ||
| self._allocations = set() # type: Set[OrderLine] | ||
| def __repr__(self): | ||
| return f"<Batch{self.reference}>" | ||
| def __eq__(self, other): | ||
| if not isinstance(other, Batch): | ||
| return False | ||
| return other.reference == self.reference | ||
| def __hash__(self): | ||
| return hash(self.reference) | ||
| def __gt__(self, other): | ||
| if self.eta is None: | ||
| return False | ||
| if other.eta is None: | ||
| return True | ||
| return self.eta > other.eta | ||
| def allocate(self, line: OrderLine): | ||
| if self.can_allocate(line): | ||
| self._allocations.add(line) | ||
| def deallocate(self, line: OrderLine): | ||
| if line in self._allocations: | ||
| self._allocations.remove(line) | ||
| @property | ||
| def allocated_quantity(self) -> int: | ||
| return sum(line.qty for line in self._allocations) | ||
| @property | ||
| def available_quantity(self) -> int: | ||
| return self._purchased_quantity - self.allocated_quantity | ||
| def can_allocate(self, line: OrderLine) -> bool: | ||
| return self.sku == line.sku and self.available_quantity >= line.qty | ||
Empty file.
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.