[core] Add Client Side Scraping of Sources (#118)

It is now possible to add and update sources via client side scraping.
For that a new edge function `add-or-update-source-v1` was added and the
old `add-source-v1` function was deprecated.

The new function accepts a new `feedData` field, which can contain the
feed for a source. If the field is provided we will not try to get the
feed for a source within our edge function and instead use the provided
data.

Currently this function is only used to add a Reddit source. Later we
plan to extend it for other sources and want to use it to update source
via the app, when the source provider makes heavy use of rate limiting.
This commit is contained in:
Rico Berger
2024-01-30 21:59:42 +01:00
committed by GitHub
parent fac622ef97
commit 90fc7532ba
39 changed files with 583 additions and 54 deletions

View File

@@ -87,6 +87,7 @@ jobs:
supabase db push
supabase functions deploy add-or-update-source-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy add-source-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy delete-user-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy generate-magic-link-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
@@ -109,6 +110,7 @@ jobs:
supabase db push
supabase functions deploy add-or-update-source-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy add-source-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy delete-user-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json
supabase functions deploy generate-magic-link-v1 --project-ref $PROJECT_ID --import-map supabase/functions/import_map.json