The Wayback Machine - https://web.archive.org/web/20220207143645/https://github.com/Graylog2/graylog2-server/issues/6502
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Discrepancy in display counts on Sources view #6502

Open
gimmic opened this issue Sep 27, 2019 · 2 comments
Open

Discrepancy in display counts on Sources view #6502

gimmic opened this issue Sep 27, 2019 · 2 comments
Labels
feature good first issue triaged

Comments

@gimmic
Copy link

@gimmic gimmic commented Sep 27, 2019 •
edited

I have noticed when ingesting backlog(older timestamped data) that the "Messages per minute" line graph and "sources" data do not line up.

The Messages per minute appear to be correct for the ingest rate, but the sources breakdown below it only show messages for each type from within the time window via timestamp. This means in the last hour if you've ingested logs from 2 days ago, the data is not represented as "sources within the last hour".

I would prefer the log sources overview to use relative time to current, not relative time to the log sources for determining sources, if that makes sense.

Expected Behavior

Ingesting logs with a timestamp of 2 days ago within the recent hour should be represented in the sources count for the relative "1 hour" window.

Current Behavior

The sources display only shows sources which have a message timestamp within the last hour. This means anything older than the relative time window is not displayed- even though it was actually ingested during the past hour.

Possible Solution

Use indexed time or another method to query sources in the last hour rather than message timestamp field.

Steps to Reproduce (for bugs)

  1. Rapidly ingest older log data
  2. Look at sources page.

Your Environment

  • Graylog Version: 3.1.2
  • Elasticsearch Version: 6.8
@kroepke
Copy link
Member

@kroepke kroepke commented Sep 30, 2019

Hi!

So, starting in 3.1 Graylog is tracking the ingest time as well, for use with alerting.
As you've noted, the sources page is currently simply aggregating on the regular timestamp, and while the medium-term plan is to replace the entire page with a dedicated dashboard, I think a quick fix could be to make the timestamp it uses configurable to support the analysis you are looking for.

@gimmic
Copy link
Author

@gimmic gimmic commented Oct 15, 2019

in relation, I have built a "better sources dashboard" that I've been using since 3.x. It would be a nice/simple fix to allow the Source page to be editable like a regular dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
feature good first issue triaged
Projects
None yet
Milestone
No milestone
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
@kroepke @gimmic