Until now it could happen that the items for a column could not be
retrieved from the database, because of the set query timeout. In this
case we received the following error:
```
PostgrestException(message: canceling statement due to statement timeout, code: 57014, details: Internal Server Error, hint: null)
```
To fix this issue we added an `items_columnId_idx` index, so that the
items for a column are retrieved fast.
We also added some other useful indexes to improve the overall
performance of our queries.
Last but not least we also added an index to the `userId` columns as it
is recommended by Supabase to improve the performance of our RLS
policies.
A 4chan source can now also be added via the RSS source type in the
Flutter app. If a user provides the RSS feed of an 4chan board, we check
if the url is related to 4chan and if this is the case, we use the 4chan
instead of the RSS source type, similar to how it is handled for other
sources like Medium, Reddit, etc.
This commit adds support for 4chan. This means that 4chan can be
selected as a new source type. When the 4chan source is selected a user
can select a board from which he wants to get the RSS feed.
Reddit and Nitter are heavlily rate limited from time to time, to not
effect our premium users, by these limits, we decided to increase the
interval in which the Reddit and Nitter sources are updated for users
which are on the free tier.
We do not remove HTML tags from the description of a RSS feed item
anymore, because:
- This didn't worked for all feeds, because we first removed the tags
and then unescape the data, which didn't make sense
- We render the HTML to Makrdown in the frontend so that the description
can contain HTML tags and is still properly rendered and in some case
better rendered then before
The log level is now configurable, via the `FEEDDECK_LOG_LEVEL`
environment variable. The environment variable can have the following
values: `debug`, `info`, `warning` or `error`.
With this change we also only log the response when getting and parsing
the feed fails when the log level is set to `debug`, also when the
actual message is still an error. This should reduce the noice in the
logs a lot and allows us to specially turn this on while debugging.
It is now possible to add and update sources via client side scraping.
For that a new edge function `add-or-update-source-v1` was added and the
old `add-source-v1` function was deprecated.
The new function accepts a new `feedData` field, which can contain the
feed for a source. If the field is provided we will not try to get the
feed for a source within our edge function and instead use the provided
data.
Currently this function is only used to add a Reddit source. Later we
plan to extend it for other sources and want to use it to update source
via the app, when the source provider makes heavy use of rate limiting.
This commit refactors the existing tools, by moving the tools logic to a
new `tools.ts` file, so that the main `cmd.ts` file remains clear.
Besides that we also add a new tool `get-feed` which can be used to run
the `getFeed` function from the command line. The function is called
with a source and returns the generated source and items, as they are
saved in the database by the `add-source-v1` Supabase edge function.
This commit improve the error handling for the edge function to add a
new source and the worker, so that we get more insights why a request
fails. A user will now also get a more detailed error why a source could
not be added.
If an RSS feed contains a video within the `attachments` field, the video
will now be added to the `options` field of the item. In the Flutter
code we then check if the video field is present in the options and show
the video instead of an image in the details view of the item.
This commit updates all used Deno modules to their latest version.
Since some of the used modules / functions were deprecated we had to
adjust our encrypt / descrypt functions and the generation of the source
and item ids, where we have to use a new md5 function.
Until now we only checked if a website contained a RSS feed which can be
used for FeedDeck. Now we are also checking if the website contains a
Atom or RDF feed when no RSS feed was found.
For this we are checking `link` tag with the `type="application/atom+xml"`
attribute or a link tag with the `type="application/rdf+xml"` attribute.
This commit adds tests for all available sources.
This commit also fixes the parsing of Atom feeds for the RSS source,
where the `dc:date` field must be used for the `publishedAt` field.
This commit uses the "Continuous Integration" GitHub Action, to run
tests for the Deno code, which is used by the Supabase functions and our
Docker containers.
This commit also adds a first test so that the `deno test` command does
not fail.
This commit adds support to add Lemmy RSS feeds to FeedDeck. A user can
provide the url of an Lemmy instance, the url of a community or of an
user.
The special thing of the Lemmy source in opposite to the normal RSS
source is, that we parse the provided link form a feed item, to check if
it contains a image, video or YouTube url, to apply some special
formatting.
When using Medium as a source it is possible that the a lot of items
might be spam when following a tag. To reduce the spam, we filter these
items based on a word list and an calculated score. The score is
calculated by the number of words which are included in the title from
our word list.
When a Reddit post contained an image it was always rendered as table,
where the image was displayed in the left column and the author in the
right column. This looked very ugly in the app, so that we are now
removing all tables from the description of an Reddit post, so that the
image is rendered as large as possible and the author is displayed below
the image.
In the `getMedia` function for Medium, Nitter, RSS and Tumblr we checked
the content and/or description of an RSS feed entry for a media file,
but we didn't pass the string to the `unescape` function first, so that
we might missed some media files, because our regular expression were
not able to find an image.
Now we are using the `unescape` function before using our regular
expression to find the image, similar to how we are also applying the
`unescape` function before we save the item description.
This commit adds a new source type "pinterest", which can be used to
follow the post of an user or a board on Pinterest. To use the new
source type a user can select the "Pinterest" item in the add source
modal. In the form a user can provide the username or board he wants to
follow via FeedDeck.
In the corresponding Supabase function we then convert the input
provided by the user to an valid RSS feed url for Pinterest. This means
that we have to add `/feed.rss` for users and `.rss` for boards to the
Pinterest url.
Then we generate the source and items as for the other sources and reuse
the existing components to render the preview and details item. We had
to adjust the rendering logic for these items, to ignore empty values,
from which also other sources will benefit.
We have to add the "charset" parameter to the "Content-Type" header when
we return json from one of the Supabase functions, so that special
characters are properly decoded in Flutter.
Users can now provide the URL of a website instead of the url of a RSS
feed via the input field for the RSS source.
This is possible because we are now trying to get and parse the RSS feed
for the provided url as usual, but if this operation fails, we try to
parse the text as html, so that we can check if it contains a
"<link type="application/rss+xml" href="RSS_FEED_URL">" tag. If this is
the case we are using this value to try to get and parse the RSS feed
again. If it fails again we are returning an error as before.
This commit improves / simplifies the media handling within the app.
Until now we always had to provide the type of the media file (item
media / source icon) when we wanted to display it. This is now not
necessary anymore. Instead we always display the image from it's
original path when the url starts with "http://" or "https://".
Additionally we also check the platform to proxy the image request on
the web. If the image doesn't start with "http://" or "https://" we
always try to display the image from the Supabase storage.
For this we also adjusted the corresponding Deno function, so that we
check if the image starts with "http://" or "https://" before we upload
it to the Supabase storage. If this is the case we upload the source
icon to the Supabase storage. If the upload fails, we will use the
original path of the icon.
Last but not least this commit also introduces our own
"CachedNetworkImage" widget, which wraps the original
"CachedNetworkImage" widget. Our own widget will ensure that we use the
correct url for the image request, so that we do not have to use this
function all over the app anymore. Later this widget can also be used to
introduce our own cache manager.
Add some stuff to make the development with Neovim easier:
- Add a "deno.json" file, which is required by the Deno language server,
when working in Neovim to detect the "import_map.json" file.
- Add a "run.sh" script to easily run the Flutter app from the command
line.
- Add the "run.sh" script and an example for Neovim to the contributing
guide.
If a notification was related to a pull request the link we added to the
item didn't work. This should now be fixed, so that when a user clicks
on the item he is redirected to the correct pull request which is
associated with the item.
This commit also removes an unnecessary "console.log" statement.
The items of an RSS feed are now rendered better, to achieve this we did
the following changes:
- Remove leading and trailing whitespaces from the item description
which should be rendered.
- Check if the media file of an item is an SVG image. If this is the
case we will not add it to the "media" field in the database, because
currently the CachedNetworkImage widget can not render SVGs. If we
want to render them, we run into serious performance issue so we skip
them completly.
- Always assume that the content of an RSS feed contains HTML and render
them as plain text in the preview and as markdown in the details.
Since we also render images from the description now, we check if the
"item.media" image should be rendered. If the description contains an
image we do not render our own image. If the description doesn't
contain a image we render it.
It is now possible to play videos from toots within FeedDeck. For that
we are using the "madia_kit" package, which is already used for the
Podcast player on Windows and Linux.
The videos from a toot are saved within the "options.videos" field of an
item next to the "options.media" field. In the "ItemDetailsMastodon"
widget we are then checking if this field is present and contains a list
of video urls. These urls can then be played via the "ItemVideos"
widget.
This commit fixes two bugs within the Podcast player.
It could happen that the play time and remaining time was not shown
correctly in the player. This mainly occured in Safari and is now fixed
by not using a regular expression to show the time, but instead we have
added a new "_printDuration" function which handles the formatting of
the durations.
Not only, but mainly on Safari it could also happen, that once a Podcast
was started and then paused, the loading spinner was displayed and a
user could not continue with the Podcast. This is now fixed, by removing
the spinner and only show the play, pause or replay button. As loading
indication we are now only using the seek bar.
Last but not least this commit also improves the readability of the
code, for parsing the icon of a Podcast feed.
This commit introduces a new "profile-v2" edge function and deprecates
the "profile-v1" edge function. We decided to provide a new edge
function for all profile related operation to improve the handling of
the different operation the edge function is responsible for.
The main differences to the "profile-v1" functions are:
- Pass the operation id within the request url instead of the body of
the request.
- Move the operations for accounts (e.g. GitHub) to seperate files, to
improve the readability of the function.
- Do not require the "sourceType" anymore, instead we are using the
operation id and request method to determine the operation which
should be executed
- The request body is now only used for the data of the corresponding
account and not for any other information like the operation id and
source information.
We have to add in-app purchases for the iOS, macOS and Android store, so
that users can also get the premium features of the app without using
Stripe for payments.
The in-app purchases are only enabled when a user uses the app with the
default Supabase environment or with the Supabase environment provided
during build time. If a user uses his own Supabase instance, he will not
be able to upgrade to the premium tier via in-app purchases.
We are using RevenueCat for in-app purchases, which automatically sends
all the events for a user to the "revenuecat-webhooks-v1" edge function.
Depending on the received event we can then upgrade / downgrade the
users profile. To be able to use RevenueCat as an additional provider to
Stripe we also had to add a new "subscriptionProvider" provider column
to the "profiles" table, which stores the information via which provider
a user upgraded his account.
For the item ingestion logic we are now also looking at the published
date for all entries in a RSS feed. If the published date is older then
the last update date of a source the item will be skipped. This way we
should reduce the number of duplicated items, where an entry was updated
in the RSS feed, so that it had a new id and was written again to the
database and not only updated.
In the "isYoutubeUrl" function we are checking if the provided url is
related to YouTube. For that we checked if the url starts with
"https://www.youtube.com/" or "https://m.youtube.com/", but we also have
to check if the url starts with "https://youtube.com/" which is used
when a users shares a link to a channel via the YouTube app.
In this commit we add the missing "https://youtube.com/" option to the
"isYoutubeUrl" function.
