- Set `search_path` for all database functions, as this is recommended
by the Supabase security advisor
- Fix `items_delete` functions: The function was never working, which
caused that we saved to many items for a source, which slowed down our
performance
- Revoke execution rights for users for the `sources_delete_files` and
`items_delete` functions
- Add E2E test, to check if users can call the database functions
Until now we had a `find` command to delete existing edge functions
before copying them for the self hosting set up. This command is now
removed, because we also moved the `main` function to the existing
functions directory and just not push it as function to Supabase.
Instead of using an `import_map.json` file to define the versions for
dependencies, they are now defined directly within the import. Since the
`import_map.json` file should not be used anymore and instead a
`deno.json` file per function should be used, we decided to define them
directly with the code. The overhead compared to a `deno.json` file per
function shouldn't be that large and it makes using functions in a
self-hosted setup easier.
Until now, we always checked the `description` field of an item first to
get our description value. If the `description` field is present we
directly used it and never checked the `content` field.
Now we are checking the `content` field first and afterwards the
`description` field. This way by default the `content` field is used as
our description. This should be more common to get the full article text
when both fields are present in an RSS feed.
When both fields are present and the `description` field is longer then
the `content` field, we will still prefer the `description` field.
* Update Deno Version to 1.45.2
Update Deno Version in the Docker image to version 1.45.2 and update the
`@supabase/supabase-js` dependency specified in the `import_map.json`
file.
**NOTES:**
- The current recommended approach for managing dependencies is using a
`deno.json` file, which is currently not working locally and returns
the following error:
```
serving the request with supabase/functions/add-or-update-source-v1
worker boot error: failed to create the graph: Relative import path "@supabase/supabase-js" not prefixed with / or ./ or ../
at file:///Users/ricoberger/Documents/GitHub/feeddeck/feeddeck/supabase/functions/add-or-update-source-v1/index.ts:1:30
worker boot error: failed to create the graph: Relative import path "@supabase/supabase-js" not prefixed with / or ./ or ../
at file:///Users/ricoberger/Documents/GitHub/feeddeck/feeddeck/supabase/functions/add-or-update-source-v1/index.ts:1:30
InvalidWorkerCreation: worker boot error: failed to create the graph: Relative import path "@supabase/supabase-js" not prefixed with / or ./ or ../
at file:///Users/ricoberger/Documents/GitHub/feeddeck/feeddeck/supabase/functions/add-or-update-source-v1/index.ts:1:30
at async UserWorker.create (ext:sb_user_workers/user_workers.js:139:15)
at async Object.handler (file:///root/index.ts:157:22)
at async respond (ext:sb_core_main_js/js/http.js:197:14) {
name: "InvalidWorkerCreation"
}
```
- When using Deno v2 the dependencies via the `deno.json` file are
working. To enable Deno v2 the following must be set in the
`config.toml` file:
```
[edge_runtime]
deno_version = 2
```
- Deno v2 is currently only supported locally, see
https://supabase.com/blog/supabase-edge-functions-deploy-dashboard-deno-2-1#deno-21-preview
- Once Deno v2 is supported in the hosted platform, we should switch
from the `import_map.json` to `deno.json` to manage dependencies and
update the existing dependencies.
* Fix Errors
- Replace `err.toString()` with just `err` when logging errors.
- Add new `blobToFile` function, to upload files to the Supabase
storage, otherwise we receive the following error (https://github.com/feeddeck/feeddeck/actions/runs/14546547197/job/40812707871):
```
error: TS2345 [ERROR]: Argument of type 'Blob' is not assignable to parameter of type 'FileBody'.
Type 'Blob' is missing the following properties from type 'File': lastModified, name, webkitRelativePath
file,
~~~~
at file:///home/runner/work/feeddeck/feeddeck/supabase/functions/_shared/feed/utils/uploadFile.ts:66:9
```
This commit disables the adding of new Nitter sources, because Nitter is
deprecated 😞 and adding new sources is not working anymore. Nitter
sources are also not updated anymore, because the update for Nitter
sources isn't working as well. To ensure that the Nitter sources are not
updated anymore the sources with type `nitter` are skipped in the
scheduler.
The Nitter source isn't removed completly, because we would also have to
cleanup the database and their might be items, which are bookmarked by
users.
See https://github.com/zedeus/nitter/issues/1155#issuecomment-1913361757
X (Twitter) was never supported and the code was never used to add a
source. We kept it in the hope that it will be possible again to use X
within the app. Since this isn't the case it is time to remove the code.
A 4chan source can now also be added via the RSS source type in the
Flutter app. If a user provides the RSS feed of an 4chan board, we check
if the url is related to 4chan and if this is the case, we use the 4chan
instead of the RSS source type, similar to how it is handled for other
sources like Medium, Reddit, etc.
This commit adds support for 4chan. This means that 4chan can be
selected as a new source type. When the 4chan source is selected a user
can select a board from which he wants to get the RSS feed.
Reddit and Nitter are heavlily rate limited from time to time, to not
effect our premium users, by these limits, we decided to increase the
interval in which the Reddit and Nitter sources are updated for users
which are on the free tier.
We do not remove HTML tags from the description of a RSS feed item
anymore, because:
- This didn't worked for all feeds, because we first removed the tags
and then unescape the data, which didn't make sense
- We render the HTML to Makrdown in the frontend so that the description
can contain HTML tags and is still properly rendered and in some case
better rendered then before
The log level is now configurable, via the `FEEDDECK_LOG_LEVEL`
environment variable. The environment variable can have the following
values: `debug`, `info`, `warning` or `error`.
With this change we also only log the response when getting and parsing
the feed fails when the log level is set to `debug`, also when the
actual message is still an error. This should reduce the noice in the
logs a lot and allows us to specially turn this on while debugging.
It is now possible to add and update sources via client side scraping.
For that a new edge function `add-or-update-source-v1` was added and the
old `add-source-v1` function was deprecated.
The new function accepts a new `feedData` field, which can contain the
feed for a source. If the field is provided we will not try to get the
feed for a source within our edge function and instead use the provided
data.
Currently this function is only used to add a Reddit source. Later we
plan to extend it for other sources and want to use it to update source
via the app, when the source provider makes heavy use of rate limiting.
This commit refactors the existing tools, by moving the tools logic to a
new `tools.ts` file, so that the main `cmd.ts` file remains clear.
Besides that we also add a new tool `get-feed` which can be used to run
the `getFeed` function from the command line. The function is called
with a source and returns the generated source and items, as they are
saved in the database by the `add-source-v1` Supabase edge function.
This commit improve the error handling for the edge function to add a
new source and the worker, so that we get more insights why a request
fails. A user will now also get a more detailed error why a source could
not be added.
If an RSS feed contains a video within the `attachments` field, the video
will now be added to the `options` field of the item. In the Flutter
code we then check if the video field is present in the options and show
the video instead of an image in the details view of the item.
This commit updates all used Deno modules to their latest version.
Since some of the used modules / functions were deprecated we had to
adjust our encrypt / descrypt functions and the generation of the source
and item ids, where we have to use a new md5 function.
Until now we only checked if a website contained a RSS feed which can be
used for FeedDeck. Now we are also checking if the website contains a
Atom or RDF feed when no RSS feed was found.
For this we are checking `link` tag with the `type="application/atom+xml"`
attribute or a link tag with the `type="application/rdf+xml"` attribute.
This commit adds tests for all available sources.
This commit also fixes the parsing of Atom feeds for the RSS source,
where the `dc:date` field must be used for the `publishedAt` field.
This commit uses the "Continuous Integration" GitHub Action, to run
tests for the Deno code, which is used by the Supabase functions and our
Docker containers.
This commit also adds a first test so that the `deno test` command does
not fail.
This commit adds support to add Lemmy RSS feeds to FeedDeck. A user can
provide the url of an Lemmy instance, the url of a community or of an
user.
The special thing of the Lemmy source in opposite to the normal RSS
source is, that we parse the provided link form a feed item, to check if
it contains a image, video or YouTube url, to apply some special
formatting.
When using Medium as a source it is possible that the a lot of items
might be spam when following a tag. To reduce the spam, we filter these
items based on a word list and an calculated score. The score is
calculated by the number of words which are included in the title from
our word list.
When a Reddit post contained an image it was always rendered as table,
where the image was displayed in the left column and the author in the
right column. This looked very ugly in the app, so that we are now
removing all tables from the description of an Reddit post, so that the
image is rendered as large as possible and the author is displayed below
the image.
In the `getMedia` function for Medium, Nitter, RSS and Tumblr we checked
the content and/or description of an RSS feed entry for a media file,
but we didn't pass the string to the `unescape` function first, so that
we might missed some media files, because our regular expression were
not able to find an image.
Now we are using the `unescape` function before using our regular
expression to find the image, similar to how we are also applying the
`unescape` function before we save the item description.
This commit adds a new source type "pinterest", which can be used to
follow the post of an user or a board on Pinterest. To use the new
source type a user can select the "Pinterest" item in the add source
modal. In the form a user can provide the username or board he wants to
follow via FeedDeck.
In the corresponding Supabase function we then convert the input
provided by the user to an valid RSS feed url for Pinterest. This means
that we have to add `/feed.rss` for users and `.rss` for boards to the
Pinterest url.
Then we generate the source and items as for the other sources and reuse
the existing components to render the preview and details item. We had
to adjust the rendering logic for these items, to ignore empty values,
from which also other sources will benefit.
We have to add the "charset" parameter to the "Content-Type" header when
we return json from one of the Supabase functions, so that special
characters are properly decoded in Flutter.
Users can now provide the URL of a website instead of the url of a RSS
feed via the input field for the RSS source.
This is possible because we are now trying to get and parse the RSS feed
for the provided url as usual, but if this operation fails, we try to
parse the text as html, so that we can check if it contains a
"<link type="application/rss+xml" href="RSS_FEED_URL">" tag. If this is
the case we are using this value to try to get and parse the RSS feed
again. If it fails again we are returning an error as before.
This commit improves / simplifies the media handling within the app.
Until now we always had to provide the type of the media file (item
media / source icon) when we wanted to display it. This is now not
necessary anymore. Instead we always display the image from it's
original path when the url starts with "http://" or "https://".
Additionally we also check the platform to proxy the image request on
the web. If the image doesn't start with "http://" or "https://" we
always try to display the image from the Supabase storage.
For this we also adjusted the corresponding Deno function, so that we
check if the image starts with "http://" or "https://" before we upload
it to the Supabase storage. If this is the case we upload the source
icon to the Supabase storage. If the upload fails, we will use the
original path of the icon.
Last but not least this commit also introduces our own
"CachedNetworkImage" widget, which wraps the original
"CachedNetworkImage" widget. Our own widget will ensure that we use the
correct url for the image request, so that we do not have to use this
function all over the app anymore. Later this widget can also be used to
introduce our own cache manager.
Add some stuff to make the development with Neovim easier:
- Add a "deno.json" file, which is required by the Deno language server,
when working in Neovim to detect the "import_map.json" file.
- Add a "run.sh" script to easily run the Flutter app from the command
line.
- Add the "run.sh" script and an example for Neovim to the contributing
guide.
If a notification was related to a pull request the link we added to the
item didn't work. This should now be fixed, so that when a user clicks
on the item he is redirected to the correct pull request which is
associated with the item.
This commit also removes an unnecessary "console.log" statement.
The items of an RSS feed are now rendered better, to achieve this we did
the following changes:
- Remove leading and trailing whitespaces from the item description
which should be rendered.
- Check if the media file of an item is an SVG image. If this is the
case we will not add it to the "media" field in the database, because
currently the CachedNetworkImage widget can not render SVGs. If we
want to render them, we run into serious performance issue so we skip
them completly.
- Always assume that the content of an RSS feed contains HTML and render
them as plain text in the preview and as markdown in the details.
Since we also render images from the description now, we check if the
"item.media" image should be rendered. If the description contains an
image we do not render our own image. If the description doesn't
contain a image we render it.
It is now possible to play videos from toots within FeedDeck. For that
we are using the "madia_kit" package, which is already used for the
Podcast player on Windows and Linux.
The videos from a toot are saved within the "options.videos" field of an
item next to the "options.media" field. In the "ItemDetailsMastodon"
widget we are then checking if this field is present and contains a list
of video urls. These urls can then be played via the "ItemVideos"
widget.
This commit fixes two bugs within the Podcast player.
It could happen that the play time and remaining time was not shown
correctly in the player. This mainly occured in Safari and is now fixed
by not using a regular expression to show the time, but instead we have
added a new "_printDuration" function which handles the formatting of
the durations.
Not only, but mainly on Safari it could also happen, that once a Podcast
was started and then paused, the loading spinner was displayed and a
user could not continue with the Podcast. This is now fixed, by removing
the spinner and only show the play, pause or replay button. As loading
indication we are now only using the seek bar.
Last but not least this commit also improves the readability of the
code, for parsing the icon of a Podcast feed.
This commit introduces a new "profile-v2" edge function and deprecates
the "profile-v1" edge function. We decided to provide a new edge
function for all profile related operation to improve the handling of
the different operation the edge function is responsible for.
The main differences to the "profile-v1" functions are:
- Pass the operation id within the request url instead of the body of
the request.
- Move the operations for accounts (e.g. GitHub) to seperate files, to
improve the readability of the function.
- Do not require the "sourceType" anymore, instead we are using the
operation id and request method to determine the operation which
should be executed
- The request body is now only used for the data of the corresponding
account and not for any other information like the operation id and
source information.
