Skip to content

GPU stream implementation for ONNX runtime #14117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
davidrohr merged 51 commits into from
Apr 20, 2025
Merged

GPU stream implementation for ONNX runtime #14117

Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
84eac06
Initial set of bug.fixes and cosmetic changes
ChSonnabend Mar 15, 2025
2191649
Please consider the following formatting changes
alibuild Mar 15, 2025
5be779c
Merge pull request #18 from alibuild/alibot-cleanup-14069
ChSonnabend Mar 15, 2025
b742c50
Adjusting eval sizes. Makes code neater and avoids some calculations
ChSonnabend Mar 15, 2025
c0bc918
Merge branch 'dev' into gpu_clusterizer_bug_fixes
ChSonnabend Mar 19, 2025
0c1cfb7
Adding separate functions. Now the host process only needs one instan…
ChSonnabend Mar 20, 2025
83c004f
First version of CCDB implementation
ChSonnabend Mar 22, 2025
d767ed1
Working CCDB API calls (tested with test-ccdb)
ChSonnabend Mar 23, 2025
ad4b22b
Improve fetching, but have to pass settings by value, not const ref
ChSonnabend Mar 24, 2025
81c646b
Using const ref and moving CCDB calls to host initialization
ChSonnabend Mar 24, 2025
566ddb7
Simplifications and renaming
ChSonnabend Mar 25, 2025
a9c33b5
Please consider the following formatting changes
alibuild Mar 25, 2025
0ed7d25
Merge pull request #19 from alibuild/alibot-cleanup-14069
ChSonnabend Mar 25, 2025
9037ea6
First version of GPU stream implementation. Still needs testing.
ChSonnabend Mar 27, 2025
64c19d5
Fixes
ChSonnabend Mar 27, 2025
8a5bb69
Please consider the following formatting changes
alibuild Mar 27, 2025
e657928
Merge pull request #20 from alibuild/alibot-cleanup-14117
ChSonnabend Mar 27, 2025
46fb1e1
Adding the lane variable. This PR will in any case conflict with #14069
ChSonnabend Mar 27, 2025
70320c3
Compiles on EPNs. Need to add shadow processors next. But for this, I…
ChSonnabend Mar 29, 2025
3174e39
Merge branch 'gpu_clusterizer_bug_fixes' into onnx_gpu_implementation
ChSonnabend Mar 29, 2025
9d9267f
Adding shadow instance. Not sure if this correctly allocates GPU memo…
ChSonnabend Mar 29, 2025
007a4a1
This runs, but will eventually fill up the VRAM. Need to include a me…
ChSonnabend Apr 1, 2025
4ef35fc
Found the stream allocation issue. Now starting optimizations
ChSonnabend Apr 1, 2025
4faaa4a
Improve readability and adapt for some comments
ChSonnabend Apr 1, 2025
2801c2e
Fixing memory assignment issue. Reconstruction runs through with FP32…
ChSonnabend Apr 2, 2025
1dcb1da
Major reworkings to add FP16 support
ChSonnabend Apr 2, 2025
7da3793
Merge branch 'dev' into onnx_gpu_implementation
ChSonnabend Apr 2, 2025
381955a
Bug-fixes
ChSonnabend Apr 3, 2025
19b5bd5
Improved data filling speeds by factor 3
ChSonnabend Apr 3, 2025
83d0257
Limiting threads for ONNX evaluation
ChSonnabend Apr 3, 2025
fff6dc3
Bug-fix for correct thread assignment and input data filling
ChSonnabend Apr 3, 2025
b437e38
Minor changes
ChSonnabend Apr 4, 2025
710993a
Adding I** inference, potentally needed for CNN + FC inference
ChSonnabend Apr 5, 2025
77c1691
CCDB fetching of NNs ported to GPUWorkflowSpec
ChSonnabend Apr 7, 2025
a985798
Adjusting CPU threads and ORT copmile definitions
ChSonnabend Apr 10, 2025
fb08f18
About 10x speed-up due to explicit io binding
ChSonnabend Apr 10, 2025
b1c88f0
Changes for synchronization and consistency. No performance loss.
ChSonnabend Apr 11, 2025
32cab70
Please consider the following formatting changes
alibuild Apr 11, 2025
5f741fc
Merge pull request #21 from alibuild/alibot-cleanup-14117
ChSonnabend Apr 11, 2025
70907aa
Fixing warnings (errors due to size_t)
ChSonnabend Apr 11, 2025
e46cdfa
Fixing linker issues
ChSonnabend Apr 13, 2025
37955fa
Merge branch 'dev' into onnx_gpu_implementation
ChSonnabend Apr 15, 2025
4b0825a
Adding volatile memory allocation and MockedOrtAllocator. Removing pr…
ChSonnabend Apr 16, 2025
497a9d4
Please consider the following formatting changes
alibuild Apr 16, 2025
aabddb7
Merge pull request #22 from alibuild/alibot-cleanup-14117
ChSonnabend Apr 16, 2025
cfdc15f
Merge dev + fixes
ChSonnabend Apr 16, 2025
a67b634
Circumvent "unused result" warning and build failure
ChSonnabend Apr 16, 2025
938a1ed
Adjust for comments
ChSonnabend Apr 19, 2025
7b07496
Please consider the following formatting changes
alibuild Apr 19, 2025
4d3f54d
Merge pull request #23 from alibuild/alibot-cleanup-14117
ChSonnabend Apr 19, 2025
af89c9a
Fixing build flags
ChSonnabend Apr 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Please consider the following formatting changes
  • Loading branch information
@alibuild
alibuild committed Mar 15, 2025
commit 219164923257101b6084bd97700314ea4f109d30
2 changes: 1 addition & 1 deletion GPU/GPUTracking/TPCClusterFinder/GPUTPCNNClusterizer.cxx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ void GPUTPCNNClusterizer::SetMaxData(const GPUTrackingInOutPointers& io) {}

void* GPUTPCNNClusterizer::setIOPointers(void* mem)
{
if (nnClusterizerBatchedMode > 0){
if (nnClusterizerBatchedMode > 0) {
if (nnClusterizerDtype == 0 && nnClusterizerElementSize > 0) {
computePointerWithAlignment(mem, inputData16, nnClusterizerBatchedMode * nnClusterizerElementSize);
} else if (nnClusterizerDtype == 1 && nnClusterizerElementSize > 0) {
Expand Down