SG-36677 Optimize payload by prevent unnecessary data

[carlos-villavicencio-adsk]

Summary

Remove unnecessary data in the payload when combining related queries before sending it to the server. This would improve overall performance decreasing network latency and server processing.

Affected methods

• find
• summarize
• text_search (API query_display_name_cache)

Detailed Description

The change should transform calls like:

sg.find('Asset', [['project', 'is',{'created_at': datetime.datetime(2015, 12, 16, 11, 2, 10, tzinfo), 'id': 72, 'name': 'Demo: Game', 'type': 'Project'}]]) 

as if the following had been called:

sg.find('Asset', [['project', 'is',{'id': 72, 'type': 'Project'}]]) 

This version also includes a new environmental variable to enable this feature. Please use it at initialization if you want to compare results or if you are getting any unexpected behavior.

export SHOTGUN_API_ENABLE_ENTITY_OPTIMIZATION =1

Testing

Install this v3.7.0-beta.1 version.

pip install git+https://github.com/shotgunsoftware/[email protected]

In my first tests, I added 40 fake tests to a project dict and with this feature applied, the result time decreased from 0.9726s to 0.3405s.

os.environ["SHOTGUN_API_ENABLE_ENTITY_OPTIMIZATION"] ="1"sg=shotgun_api3.Shotgun(... # Getting project fieldsproject_fields=list(sg.schema_field_read("Project").keys()) # Getting a projectsg_full_project=sg.find_one('Project', [['id', 'is', PROJECT_ID]], project_fields) # At this point, `sg_full_project` can be "hacked" with more unnecessary fields to simulate a larger payload.# Getting an assetsg_asset=sg.find('Asset', [['project', 'is', sg_full_project]])

TODO

• Publish v3.7.0-beta.1 tag for testing

[julien-lang]

Another comment. Although not a blocker.

[julien-lang]

Could you update the env var name in the description too please?

It's still says:

export SHOTGUN_API_DISABLE_ENTITY_OPTIMIZATION=1

Instead of:

export SHOTGUN_API_ENABLE_ENTITY_OPTIMIZATION=1

Looking at the description some more, given that the optimization is disabled by default

[julien-lang]

I don't like using global but we already use some there I'll let it go.