ValueError from oauthlib when validating query strings is raised as 500 instead of 400 HTTP error (in DRF extension)

[AetherUnbound]

Describe the bug

This issue is essentially #954, except we've encountered it when using Django Rest Framework (and the oauth2_provider extension for it).

Here's the stack trace from the exception we're seeing:

web-1 | 2024-07-26T21:01:08.714706Z [error ] request_failed [django_structlog.middlewares.request] code=500 filename=request.py func_name=process_exception ip=172.18.0.1 lineno=197 request=GET /v1/images/?q=73%%20of%20Arkansans%20think%20that%20crime%20is%20on%20the%20rise%20in%20their%20state request_id=c58a2c8f-8899-4641-989d-dc8b703e47be user_id=2
web-1 | Traceback (most recent call last):
web-1 | File "/.venv/lib/python3.11/site-packages/asgiref/sync.py", line 518, in thread_handler
web-1 | raise exc_info[1]
web-1 | File "/.venv/lib/python3.11/site-packages/django/core/handlers/base.py", line 253, in _get_response_async
web-1 | response = await wrapped_callback(
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/adrf/viewsets.py", line 120, in async_view
web-1 | return await self.dispatch(request, *args, **kwargs)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/adrf/views.py", line 73, in async_dispatch
web-1 | response = self.handle_exception(exc)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/rest_framework/views.py", line 469, in handle_exception
web-1 | self.raise_uncaught_exception(exc)
web-1 | File "/.venv/lib/python3.11/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
web-1 | raise exc
web-1 | File "/.venv/lib/python3.11/site-packages/adrf/views.py", line 57, in async_dispatch
web-1 | await sync_to_async(self.initial)(request, *args, **kwargs)
web-1 | File "/.venv/lib/python3.11/site-packages/asgiref/sync.py", line 468, in __call__
web-1 | ret = await asyncio.shield(exec_coro)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/asgiref/current_thread_executor.py", line 40, in run
web-1 | result = self.fn(*self.args, **self.kwargs)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/asgiref/sync.py", line 522, in thread_handler
web-1 | return func(*args, **kwargs)
web-1 | ^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/rest_framework/views.py", line 414, in initial
web-1 | self.perform_authentication(request)
web-1 | File "/.venv/lib/python3.11/site-packages/rest_framework/views.py", line 324, in perform_authentication
web-1 | request.user
web-1 | File "/.venv/lib/python3.11/site-packages/adrf/requests.py", line 19, in user
web-1 | self._authenticate()
web-1 | File "/.venv/lib/python3.11/site-packages/adrf/requests.py", line 49, in _authenticate
web-1 | user_auth_tuple = authenticator.authenticate(self)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/api/conf/oauth2_extensions.py", line 21, in authenticate
web-1 | user_auth_tuple = super().authenticate(request)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauth2_provider/contrib/rest_framework/authentication.py", line 27, in authenticate
web-1 | valid, r = oauthlib_core.verify_request(request, scopes=[])
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauth2_provider/oauth2_backends.py", line 202, in verify_request
web-1 | valid, r = self.server.verify_request(uri, http_method, body, headers, scopes=scopes)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauthlib/oauth2/rfc6749/endpoints/base.py", line 112, in wrapper
web-1 | return f(endpoint, uri, *args, **kwargs)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauthlib/oauth2/rfc6749/endpoints/resource.py", line 65, in verify_request
web-1 | request = Request(uri, http_method, body, headers)
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauthlib/common.py", line 393, in __init__
web-1 | self._params.update(dict(urldecode(self.uri_query)))
web-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
web-1 | File "/.venv/lib/python3.11/site-packages/oauthlib/common.py", line 122, in urldecode
web-1 | raise ValueError('Invalid hex encoding in query string.')
web-1 | ValueError: Invalid hex encoding in query string.

To Reproduce

I'm not able to give a minimum set of steps to reproduce, but it's essentially caused by the same circumstances as #954. You can see similar instructions in WordPress/openverse#3199.

Expected behavior

Similar to the solution added in #963, this should probably also raise a SuspiciousOperation exception instead.

Version

• I have tested with the latest published release and it's still a problem.
• I have tested with the master branch and it's still a problem.

Additional context

I think we'd just need to apply the same logic from backends.py (edited in #963):

https://github.com/jazzband/django-oauth-toolkit/blob/102c85141ec44549e17080c676292e79e5eb46cc/oauth2_provider/backends.py#L18-L27

To the authentication.py module of the rest_framework extension:

https://github.com/jazzband/django-oauth-toolkit/blob/102c85141ec44549e17080c676292e79e5eb46cc/oauth2_provider/contrib/rest_framework/authentication.py#L21-L31

I will try to get a PR up for this!