Skip to content

fix: bugs in TRT 10 upgrade#2832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zewenli98 merged 2 commits into from
May 18, 2024
Merged

fix: bugs in TRT 10 upgrade #2832

zewenli98 merged 2 commits into from
May 18, 2024

Conversation

@zewenli98
Copy link
Collaborator

@zewenli98zewenli98 commented May 14, 2024

Description

Fix bugs in TRT 10 upgrade.

Fixes#2811

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@zewenli98zewenli98 self-assigned this May 14, 2024
@github-actionsgithub-actionsbot added component: conversion Issues re: Conversion stage component: api [Python] Issues re: Python API component: runtime component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels May 14, 2024
@github-actionsgithub-actionsbot requested a review from gs-oliveMay 14, 2024 00:40
github-actions[bot]

This comment was marked as resolved.

Sign in to view

github-actions[bot]

This comment was marked as resolved.

Sign in to view

github-actions[bot]

This comment was marked as resolved.

Sign in to view

@zewenli98zewenli98force-pushed the fix_trt_10_ea_upgrade_bugs branch from fa01628 to 62673bdCompareMay 14, 2024 00:45
peri044
peri044 approved these changes May 14, 2024
View reviewed changes
Copy link
Collaborator

@peri044peri044 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

narendasan
narendasan reviewed May 14, 2024
View reviewed changes
py/torch_tensorrt/dynamo/runtime/_PythonTorchTensorRTModule.py Outdated
def__init__(
self,
engine: trt.ICudaEngine,
engine: trt.tensorrt.IHostMemory,
Copy link
Collaborator

@narendasannarendasanMay 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets use the bytes interface (may be just changing the type annotation), but we use the same thing for the C++ runtime

Copy link
CollaboratorAuthor

@zewenli98zewenli98May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Currently, the engine actually saves serialized engine since TRT 10 changes API that uses builder.build_serialized_network. Do you recommend saving deserialized engines or serialized engines?

@peri044
Copy link
Collaborator

peri044 commented May 17, 2024

Can this be merged @zewenli98 ? If so, please raise a cherry pick to release/2.3 it it's needed

@zewenli98zewenli98 merged commit 100a6d7 into mainMay 18, 2024
zewenli98 added a commit that referenced this pull request May 18, 2024
@zewenli98
cherry pick of#2832
7fafea4
@zewenli98zewenli98 mentioned this pull request May 18, 2024
Merged
7 tasks
zewenli98 added a commit that referenced this pull request May 24, 2024
@zewenli98
cherry pick of#2832
c1eb9c3
laikhtewari pushed a commit that referenced this pull request May 24, 2024
@zewenli98@laikhtewari
fix: bugs in TRT 10 upgrade (#2832)
78ceda5
zewenli98 added a commit that referenced this pull request May 29, 2024
@zewenli98
chore: cherry pick of#2832(#2852)
856f33d
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@narendasannarendasannarendasan left review comments

@github-actionsgithub-actions[bot]github-actions[bot] requested changes

@peri044peri044peri044 approved these changes

@gs-olivegs-oliveAwaiting requested review from gs-olive

Assignees

@zewenli98zewenli98

Labels

cla signedcomponent: api [Python]Issues re: Python APIcomponent: conversionIssues re: Conversion stagecomponent: dynamoIssues relating to the `torch.compile` or `torch._dynamo.export` pathscomponent: runtime

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

🐛 [Bug] An assertion error when upgrading to enqueueV3 interface

5 participants

@zewenli98@peri044@narendasan@facebook-github-bot