For the item ingestion logic we are now also looking at the published date for all entries in a RSS feed. If the published date is older then the last update date of a source the item will be skipped. This way we should reduce the number of duplicated items, where an entry was updated in the RSS feed, so that it had a new id and was written again to the database and not only updated.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.
## 📋 Pull Request Information
**Original PR:** https://github.com/feeddeck/feeddeck/pull/21
**Author:** [@ricoberger](https://github.com/ricoberger)
**Created:** 10/8/2023
**Status:** ✅ Merged
**Merged:** 10/8/2023
**Merged by:** [@ricoberger](https://github.com/ricoberger)
**Base:** `main` ← **Head:** `core-use-published-date-for-item-ingestion`
---
### 📝 Commits (1)
- [`07d2985`](https://github.com/feeddeck/feeddeck/commit/07d29854b4b1cf6dfa0f174a7eee8c8d7c134952) [core] Use Published Date for Item Ingestion
### 📊 Changes
**10 files changed** (+362 additions, -152 deletions)
<details>
<summary>View changed files</summary>
📝 `supabase/functions/_shared/feed/googlenews.ts` (+36 -16)
📝 `supabase/functions/_shared/feed/mastodon.ts` (+32 -14)
📝 `supabase/functions/_shared/feed/medium.ts` (+36 -16)
📝 `supabase/functions/_shared/feed/nitter.ts` (+36 -16)
📝 `supabase/functions/_shared/feed/podcast.ts` (+33 -9)
📝 `supabase/functions/_shared/feed/reddit.ts` (+36 -16)
📝 `supabase/functions/_shared/feed/rss.ts` (+44 -17)
📝 `supabase/functions/_shared/feed/stackoverflow.ts` (+37 -16)
📝 `supabase/functions/_shared/feed/tumblr.ts` (+36 -16)
📝 `supabase/functions/_shared/feed/youtube.ts` (+36 -16)
</details>
### 📄 Description
For the item ingestion logic we are now also looking at the published date for all entries in a RSS feed. If the published date is older then the last update date of a source the item will be skipped. This way we should reduce the number of duplicated items, where an entry was updated in the RSS feed, so that it had a new id and was written again to the database and not only updated.
<!--
Keep PR title verbose enough and add prefix telling about what source it touches e.g "[rss] Add feature xyz" or if the
the PR is not realated to a source use "[core]", e.g. "[core] Fix xyz".
If you add a breaking change within your PR you should add ":warning:" to the title,
e.g. ":warning: [core] My breaking change"
-->
<!--
Description of what have been changed. Please also reference an issue, when available.
-->
---
<sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
📋 Pull Request Information
Original PR: https://github.com/feeddeck/feeddeck/pull/21
Author: @ricoberger
Created: 10/8/2023
Status: ✅ Merged
Merged: 10/8/2023
Merged by: @ricoberger
Base:
main← Head:core-use-published-date-for-item-ingestion📝 Commits (1)
07d2985[core] Use Published Date for Item Ingestion📊 Changes
10 files changed (+362 additions, -152 deletions)
View changed files
📝
supabase/functions/_shared/feed/googlenews.ts(+36 -16)📝
supabase/functions/_shared/feed/mastodon.ts(+32 -14)📝
supabase/functions/_shared/feed/medium.ts(+36 -16)📝
supabase/functions/_shared/feed/nitter.ts(+36 -16)📝
supabase/functions/_shared/feed/podcast.ts(+33 -9)📝
supabase/functions/_shared/feed/reddit.ts(+36 -16)📝
supabase/functions/_shared/feed/rss.ts(+44 -17)📝
supabase/functions/_shared/feed/stackoverflow.ts(+37 -16)📝
supabase/functions/_shared/feed/tumblr.ts(+36 -16)📝
supabase/functions/_shared/feed/youtube.ts(+36 -16)📄 Description
For the item ingestion logic we are now also looking at the published date for all entries in a RSS feed. If the published date is older then the last update date of a source the item will be skipped. This way we should reduce the number of duplicated items, where an entry was updated in the RSS feed, so that it had a new id and was written again to the database and not only updated.
🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.