Skip to content

fix: CGO Memory leak issue in GO Feature server#4291

Merged
franciscojavierarceo merged 1 commit intofeast-dev:masterfrom
EXPEbdodla:fix_memory_leak
Jun 29, 2024
Merged

fix: CGO Memory leak issue in GO Feature server#4291
franciscojavierarceo merged 1 commit intofeast-dev:masterfrom
EXPEbdodla:fix_memory_leak

Conversation

@EXPEbdodla
Copy link
Contributor

@EXPEbdodla EXPEbdodla commented Jun 17, 2024

What this PR does / why we need it:

Releases the memory used by CGOArrowAllocator. Not releasing the memory causing the memory leaks. Pods restart after all the memory allocated by container is filled up.

Which issue(s) this PR fixes:

NA

Fixes

  • Memory leak in GO feature server which is causing the pod restarts after allocated memory filled up
  • Sets the response content type to application/json (Before its application/text type)
  • Added health check endpoint for GO HTTP feature server

@EXPEbdodla
Copy link
Contributor Author

EXPEbdodla commented Jun 19, 2024

pr-integration-tests failure is not related to this code change.

@tokoko
Copy link
Collaborator

tokoko commented Jun 20, 2024

You can either make some dummy commit or maybe one of the maintainers can force a rerun? @franciscojavierarceo @HaoXuAI

@tokoko
Copy link
Collaborator

tokoko commented Jun 25, 2024

@EXPEbdodla All commits need a sign-off. Follow instructions here, git rebase HEAD~3 --signoff and then git push --force-with-lease origin fix_memory_leak.

tokoko
tokoko approved these changes Jun 26, 2024
View reviewed changes
Copy link
Collaborator

@tokoko tokoko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@franciscojavierarceo franciscojavierarceo merged commit 43e198f into feast-dev:master Jun 29, 2024
tsisodia10 pushed a commit to tsisodia10/feast that referenced this pull request Jul 23, 2024
@EXPEbdodla @tsisodia10
fix: CGO Memory leak issue in GO Feature server (feast-dev#4291)
cbe8491
tsisodia10 pushed a commit to tsisodia10/feast that referenced this pull request Jul 23, 2024
@EXPEbdodla @tsisodia10
fix: CGO Memory leak issue in GO Feature server (feast-dev#4291)
d1acfdb
nick-amaya-sp pushed a commit to sailpoint/feast that referenced this pull request Jul 23, 2024
@EXPEbdodla @nick-amaya-sp
fix: CGO Memory leak issue in GO Feature server (feast-dev#4291)
5a8edc5
franciscojavierarceo pushed a commit that referenced this pull request Jul 31, 2024
@feast-ci-bot
chore(release): release 0.40.0
32ec0e0 
# [0.40.0](v0.39.0...v0.40.0) (2024-07-31)

### Bug Fixes

* Added missing type ([#4315](#4315)) ([86af60a](86af60a))
* Avoid XSS attack from Jinjin2's Environment(). ([#4355](#4355)) ([40270e7](40270e7))
* CGO Memory leak issue in GO Feature server ([#4291](#4291)) ([43e198f](43e198f))
* Deprecated the datetime.utcfromtimestamp(). ([#4306](#4306)) ([21deec8](21deec8))
* Fix SQLite import issue ([#4294](#4294)) ([398ea3b](398ea3b))
* Increment operator to v0.39.0 ([#4368](#4368)) ([3ddb4fb](3ddb4fb))
* Minor typo in the unit test. ([#4296](#4296)) ([6c75e84](6c75e84))
* OnDemandFeatureView type inference for array types ([#4310](#4310)) ([c45ff72](c45ff72))
* Remove redundant batching in PostgreSQLOnlineStore.online_write_batch and fix progress bar ([#4331](#4331)) ([0d89d15](0d89d15))
* Remove typo. ([#4351](#4351)) ([92d17de](92d17de))
* Retire the datetime.utcnow(). ([#4352](#4352)) ([a8bc696](a8bc696))
* Update dask version to support pandas 1.x ([#4326](#4326)) ([a639d61](a639d61))
* Update Feast object metadata in the registry ([#4257](#4257)) ([8028ae0](8028ae0))
* Using one single function call for utcnow(). ([#4307](#4307)) ([98ff63c](98ff63c))

### Features

* Add async feature retrieval for Postgres Online Store ([#4327](#4327)) ([cea52e9](cea52e9))
* Add Async refresh to Sql Registry ([#4251](#4251)) ([f569786](f569786))
* Add SingleStore as an OnlineStore ([#4285](#4285)) ([2c38946](2c38946))
* Add Tornike to maintainers.md ([#4339](#4339)) ([8e8c1f2](8e8c1f2))
* Bump psycopg2 to psycopg3 for all Postgres components ([#4303](#4303)) ([9451d9c](9451d9c))
* Entity key deserialization ([#4284](#4284)) ([83fad15](83fad15))
* Ignore paths feast apply ([#4276](#4276)) ([b4d54af](b4d54af))
* Move get_online_features to OnlineStore interface ([#4319](#4319)) ([7072fd0](7072fd0))
* Port mssql contrib offline store to ibis ([#4360](#4360)) ([7914cbd](7914cbd))

### Reverts

* Revert "fix: Avoid XSS attack from Jinjin2's Environment()." ([#4357](#4357)) ([cdeab48](cdeab48)), closes [#4355](#4355)
redhatHameed pushed a commit to RHEcosystemAppEng/feast that referenced this pull request Aug 5, 2024
@feast-ci-bot @redhatHameed
chore(release): release 0.40.0
cd6bbdd 
# [0.40.0](feast-dev/feast@v0.39.0...v0.40.0) (2024-07-31)

### Bug Fixes

* Added missing type ([feast-dev#4315](feast-dev#4315)) ([86af60a](feast-dev@86af60a))
* Avoid XSS attack from Jinjin2's Environment(). ([feast-dev#4355](feast-dev#4355)) ([40270e7](feast-dev@40270e7))
* CGO Memory leak issue in GO Feature server ([feast-dev#4291](feast-dev#4291)) ([43e198f](feast-dev@43e198f))
* Deprecated the datetime.utcfromtimestamp(). ([feast-dev#4306](feast-dev#4306)) ([21deec8](feast-dev@21deec8))
* Fix SQLite import issue ([feast-dev#4294](feast-dev#4294)) ([398ea3b](feast-dev@398ea3b))
* Increment operator to v0.39.0 ([feast-dev#4368](feast-dev#4368)) ([3ddb4fb](feast-dev@3ddb4fb))
* Minor typo in the unit test. ([feast-dev#4296](feast-dev#4296)) ([6c75e84](feast-dev@6c75e84))
* OnDemandFeatureView type inference for array types ([feast-dev#4310](feast-dev#4310)) ([c45ff72](feast-dev@c45ff72))
* Remove redundant batching in PostgreSQLOnlineStore.online_write_batch and fix progress bar ([feast-dev#4331](feast-dev#4331)) ([0d89d15](feast-dev@0d89d15))
* Remove typo. ([feast-dev#4351](feast-dev#4351)) ([92d17de](feast-dev@92d17de))
* Retire the datetime.utcnow(). ([feast-dev#4352](feast-dev#4352)) ([a8bc696](feast-dev@a8bc696))
* Update dask version to support pandas 1.x ([feast-dev#4326](feast-dev#4326)) ([a639d61](feast-dev@a639d61))
* Update Feast object metadata in the registry ([feast-dev#4257](feast-dev#4257)) ([8028ae0](feast-dev@8028ae0))
* Using one single function call for utcnow(). ([feast-dev#4307](feast-dev#4307)) ([98ff63c](feast-dev@98ff63c))

### Features

* Add async feature retrieval for Postgres Online Store ([feast-dev#4327](feast-dev#4327)) ([cea52e9](feast-dev@cea52e9))
* Add Async refresh to Sql Registry ([feast-dev#4251](feast-dev#4251)) ([f569786](feast-dev@f569786))
* Add SingleStore as an OnlineStore ([feast-dev#4285](feast-dev#4285)) ([2c38946](feast-dev@2c38946))
* Add Tornike to maintainers.md ([feast-dev#4339](feast-dev#4339)) ([8e8c1f2](feast-dev@8e8c1f2))
* Bump psycopg2 to psycopg3 for all Postgres components ([feast-dev#4303](feast-dev#4303)) ([9451d9c](feast-dev@9451d9c))
* Entity key deserialization ([feast-dev#4284](feast-dev#4284)) ([83fad15](feast-dev@83fad15))
* Ignore paths feast apply ([feast-dev#4276](feast-dev#4276)) ([b4d54af](feast-dev@b4d54af))
* Move get_online_features to OnlineStore interface ([feast-dev#4319](feast-dev#4319)) ([7072fd0](feast-dev@7072fd0))
* Port mssql contrib offline store to ibis ([feast-dev#4360](feast-dev#4360)) ([7914cbd](feast-dev@7914cbd))

### Reverts

* Revert "fix: Avoid XSS attack from Jinjin2's Environment()." ([feast-dev#4357](feast-dev#4357)) ([cdeab48](feast-dev@cdeab48)), closes [feast-dev#4355](feast-dev#4355)
shuchu pushed a commit to shuchu/feast that referenced this pull request Aug 14, 2024
@feast-ci-bot @shuchu
chore(release): release 0.40.0
cae788f 
# [0.40.0](feast-dev/feast@v0.39.0...v0.40.0) (2024-07-31)

### Bug Fixes

* Added missing type ([feast-dev#4315](feast-dev#4315)) ([86af60a](feast-dev@86af60a))
* Avoid XSS attack from Jinjin2's Environment(). ([feast-dev#4355](feast-dev#4355)) ([40270e7](feast-dev@40270e7))
* CGO Memory leak issue in GO Feature server ([feast-dev#4291](feast-dev#4291)) ([43e198f](feast-dev@43e198f))
* Deprecated the datetime.utcfromtimestamp(). ([feast-dev#4306](feast-dev#4306)) ([21deec8](feast-dev@21deec8))
* Fix SQLite import issue ([feast-dev#4294](feast-dev#4294)) ([398ea3b](feast-dev@398ea3b))
* Increment operator to v0.39.0 ([feast-dev#4368](feast-dev#4368)) ([3ddb4fb](feast-dev@3ddb4fb))
* Minor typo in the unit test. ([feast-dev#4296](feast-dev#4296)) ([6c75e84](feast-dev@6c75e84))
* OnDemandFeatureView type inference for array types ([feast-dev#4310](feast-dev#4310)) ([c45ff72](feast-dev@c45ff72))
* Remove redundant batching in PostgreSQLOnlineStore.online_write_batch and fix progress bar ([feast-dev#4331](feast-dev#4331)) ([0d89d15](feast-dev@0d89d15))
* Remove typo. ([feast-dev#4351](feast-dev#4351)) ([92d17de](feast-dev@92d17de))
* Retire the datetime.utcnow(). ([feast-dev#4352](feast-dev#4352)) ([a8bc696](feast-dev@a8bc696))
* Update dask version to support pandas 1.x ([feast-dev#4326](feast-dev#4326)) ([a639d61](feast-dev@a639d61))
* Update Feast object metadata in the registry ([feast-dev#4257](feast-dev#4257)) ([8028ae0](feast-dev@8028ae0))
* Using one single function call for utcnow(). ([feast-dev#4307](feast-dev#4307)) ([98ff63c](feast-dev@98ff63c))

### Features

* Add async feature retrieval for Postgres Online Store ([feast-dev#4327](feast-dev#4327)) ([cea52e9](feast-dev@cea52e9))
* Add Async refresh to Sql Registry ([feast-dev#4251](feast-dev#4251)) ([f569786](feast-dev@f569786))
* Add SingleStore as an OnlineStore ([feast-dev#4285](feast-dev#4285)) ([2c38946](feast-dev@2c38946))
* Add Tornike to maintainers.md ([feast-dev#4339](feast-dev#4339)) ([8e8c1f2](feast-dev@8e8c1f2))
* Bump psycopg2 to psycopg3 for all Postgres components ([feast-dev#4303](feast-dev#4303)) ([9451d9c](feast-dev@9451d9c))
* Entity key deserialization ([feast-dev#4284](feast-dev#4284)) ([83fad15](feast-dev@83fad15))
* Ignore paths feast apply ([feast-dev#4276](feast-dev#4276)) ([b4d54af](feast-dev@b4d54af))
* Move get_online_features to OnlineStore interface ([feast-dev#4319](feast-dev#4319)) ([7072fd0](feast-dev@7072fd0))
* Port mssql contrib offline store to ibis ([feast-dev#4360](feast-dev#4360)) ([7914cbd](feast-dev@7914cbd))

### Reverts

* Revert "fix: Avoid XSS attack from Jinjin2's Environment()." ([feast-dev#4357](feast-dev#4357)) ([cdeab48](feast-dev@cdeab48)), closes [feast-dev#4355](feast-dev#4355)
@franciscojavierarceo
Copy link
Member

franciscojavierarceo commented Aug 27, 2024

Hi @EXPEbdodla I wanted to give you a shout out in our newsletter but didn't see your name in your actual GitHub Profile, feel free to let me know or add it to the doc if you wouldn't mind here: https://docs.google.com/document/d/1sKD1Lt3h4G9vzitbgoOwNX9uOrLCjdzw4bJnUIO4ixU/edit

@EXPEbdodla
Copy link
Contributor Author

EXPEbdodla commented Aug 27, 2024

Thanks @franciscojavierarceo . Added my name to the Google Doc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@franciscojavierarceo franciscojavierarceo franciscojavierarceo approved these changes

@tokoko tokoko tokoko approved these changes

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@EXPEbdodla @tokoko @franciscojavierarceo