It is now possible to add and update sources via client side scraping.
For that a new edge function `add-or-update-source-v1` was added and the
old `add-source-v1` function was deprecated.
The new function accepts a new `feedData` field, which can contain the
feed for a source. If the field is provided we will not try to get the
feed for a source within our edge function and instead use the provided
data.
Currently this function is only used to add a Reddit source. Later we
plan to extend it for other sources and want to use it to update source
via the app, when the source provider makes heavy use of rate limiting.
This commit refactors the existing tools, by moving the tools logic to a
new `tools.ts` file, so that the main `cmd.ts` file remains clear.
Besides that we also add a new tool `get-feed` which can be used to run
the `getFeed` function from the command line. The function is called
with a source and returns the generated source and items, as they are
saved in the database by the `add-source-v1` Supabase edge function.
This commit improve the error handling for the edge function to add a
new source and the worker, so that we get more insights why a request
fails. A user will now also get a more detailed error why a source could
not be added.
If an RSS feed contains a video within the `attachments` field, the video
will now be added to the `options` field of the item. In the Flutter
code we then check if the video field is present in the options and show
the video instead of an image in the details view of the item.
This commit updates all used Deno modules to their latest version.
Since some of the used modules / functions were deprecated we had to
adjust our encrypt / descrypt functions and the generation of the source
and item ids, where we have to use a new md5 function.
Until now we only checked if a website contained a RSS feed which can be
used for FeedDeck. Now we are also checking if the website contains a
Atom or RDF feed when no RSS feed was found.
For this we are checking `link` tag with the `type="application/atom+xml"`
attribute or a link tag with the `type="application/rdf+xml"` attribute.
This commit adds tests for all available sources.
This commit also fixes the parsing of Atom feeds for the RSS source,
where the `dc:date` field must be used for the `publishedAt` field.
This commit uses the "Continuous Integration" GitHub Action, to run
tests for the Deno code, which is used by the Supabase functions and our
Docker containers.
This commit also adds a first test so that the `deno test` command does
not fail.
This commit adds support to add Lemmy RSS feeds to FeedDeck. A user can
provide the url of an Lemmy instance, the url of a community or of an
user.
The special thing of the Lemmy source in opposite to the normal RSS
source is, that we parse the provided link form a feed item, to check if
it contains a image, video or YouTube url, to apply some special
formatting.
When using Medium as a source it is possible that the a lot of items
might be spam when following a tag. To reduce the spam, we filter these
items based on a word list and an calculated score. The score is
calculated by the number of words which are included in the title from
our word list.
When a Reddit post contained an image it was always rendered as table,
where the image was displayed in the left column and the author in the
right column. This looked very ugly in the app, so that we are now
removing all tables from the description of an Reddit post, so that the
image is rendered as large as possible and the author is displayed below
the image.
In the `getMedia` function for Medium, Nitter, RSS and Tumblr we checked
the content and/or description of an RSS feed entry for a media file,
but we didn't pass the string to the `unescape` function first, so that
we might missed some media files, because our regular expression were
not able to find an image.
Now we are using the `unescape` function before using our regular
expression to find the image, similar to how we are also applying the
`unescape` function before we save the item description.
This commit adds a new source type "pinterest", which can be used to
follow the post of an user or a board on Pinterest. To use the new
source type a user can select the "Pinterest" item in the add source
modal. In the form a user can provide the username or board he wants to
follow via FeedDeck.
In the corresponding Supabase function we then convert the input
provided by the user to an valid RSS feed url for Pinterest. This means
that we have to add `/feed.rss` for users and `.rss` for boards to the
Pinterest url.
Then we generate the source and items as for the other sources and reuse
the existing components to render the preview and details item. We had
to adjust the rendering logic for these items, to ignore empty values,
from which also other sources will benefit.
We have to add the "charset" parameter to the "Content-Type" header when
we return json from one of the Supabase functions, so that special
characters are properly decoded in Flutter.
Users can now provide the URL of a website instead of the url of a RSS
feed via the input field for the RSS source.
This is possible because we are now trying to get and parse the RSS feed
for the provided url as usual, but if this operation fails, we try to
parse the text as html, so that we can check if it contains a
"<link type="application/rss+xml" href="RSS_FEED_URL">" tag. If this is
the case we are using this value to try to get and parse the RSS feed
again. If it fails again we are returning an error as before.
This commit improves / simplifies the media handling within the app.
Until now we always had to provide the type of the media file (item
media / source icon) when we wanted to display it. This is now not
necessary anymore. Instead we always display the image from it's
original path when the url starts with "http://" or "https://".
Additionally we also check the platform to proxy the image request on
the web. If the image doesn't start with "http://" or "https://" we
always try to display the image from the Supabase storage.
For this we also adjusted the corresponding Deno function, so that we
check if the image starts with "http://" or "https://" before we upload
it to the Supabase storage. If this is the case we upload the source
icon to the Supabase storage. If the upload fails, we will use the
original path of the icon.
Last but not least this commit also introduces our own
"CachedNetworkImage" widget, which wraps the original
"CachedNetworkImage" widget. Our own widget will ensure that we use the
correct url for the image request, so that we do not have to use this
function all over the app anymore. Later this widget can also be used to
introduce our own cache manager.
Add some stuff to make the development with Neovim easier:
- Add a "deno.json" file, which is required by the Deno language server,
when working in Neovim to detect the "import_map.json" file.
- Add a "run.sh" script to easily run the Flutter app from the command
line.
- Add the "run.sh" script and an example for Neovim to the contributing
guide.
If a notification was related to a pull request the link we added to the
item didn't work. This should now be fixed, so that when a user clicks
on the item he is redirected to the correct pull request which is
associated with the item.
This commit also removes an unnecessary "console.log" statement.
The items of an RSS feed are now rendered better, to achieve this we did
the following changes:
- Remove leading and trailing whitespaces from the item description
which should be rendered.
- Check if the media file of an item is an SVG image. If this is the
case we will not add it to the "media" field in the database, because
currently the CachedNetworkImage widget can not render SVGs. If we
want to render them, we run into serious performance issue so we skip
them completly.
- Always assume that the content of an RSS feed contains HTML and render
them as plain text in the preview and as markdown in the details.
Since we also render images from the description now, we check if the
"item.media" image should be rendered. If the description contains an
image we do not render our own image. If the description doesn't
contain a image we render it.
It is now possible to play videos from toots within FeedDeck. For that
we are using the "madia_kit" package, which is already used for the
Podcast player on Windows and Linux.
The videos from a toot are saved within the "options.videos" field of an
item next to the "options.media" field. In the "ItemDetailsMastodon"
widget we are then checking if this field is present and contains a list
of video urls. These urls can then be played via the "ItemVideos"
widget.
This commit fixes two bugs within the Podcast player.
It could happen that the play time and remaining time was not shown
correctly in the player. This mainly occured in Safari and is now fixed
by not using a regular expression to show the time, but instead we have
added a new "_printDuration" function which handles the formatting of
the durations.
Not only, but mainly on Safari it could also happen, that once a Podcast
was started and then paused, the loading spinner was displayed and a
user could not continue with the Podcast. This is now fixed, by removing
the spinner and only show the play, pause or replay button. As loading
indication we are now only using the seek bar.
Last but not least this commit also improves the readability of the
code, for parsing the icon of a Podcast feed.
This commit introduces a new "profile-v2" edge function and deprecates
the "profile-v1" edge function. We decided to provide a new edge
function for all profile related operation to improve the handling of
the different operation the edge function is responsible for.
The main differences to the "profile-v1" functions are:
- Pass the operation id within the request url instead of the body of
the request.
- Move the operations for accounts (e.g. GitHub) to seperate files, to
improve the readability of the function.
- Do not require the "sourceType" anymore, instead we are using the
operation id and request method to determine the operation which
should be executed
- The request body is now only used for the data of the corresponding
account and not for any other information like the operation id and
source information.
We have to add in-app purchases for the iOS, macOS and Android store, so
that users can also get the premium features of the app without using
Stripe for payments.
The in-app purchases are only enabled when a user uses the app with the
default Supabase environment or with the Supabase environment provided
during build time. If a user uses his own Supabase instance, he will not
be able to upgrade to the premium tier via in-app purchases.
We are using RevenueCat for in-app purchases, which automatically sends
all the events for a user to the "revenuecat-webhooks-v1" edge function.
Depending on the received event we can then upgrade / downgrade the
users profile. To be able to use RevenueCat as an additional provider to
Stripe we also had to add a new "subscriptionProvider" provider column
to the "profiles" table, which stores the information via which provider
a user upgraded his account.
For the item ingestion logic we are now also looking at the published
date for all entries in a RSS feed. If the published date is older then
the last update date of a source the item will be skipped. This way we
should reduce the number of duplicated items, where an entry was updated
in the RSS feed, so that it had a new id and was written again to the
database and not only updated.
In the "isYoutubeUrl" function we are checking if the provided url is
related to YouTube. For that we checked if the url starts with
"https://www.youtube.com/" or "https://m.youtube.com/", but we also have
to check if the url starts with "https://youtube.com/" which is used
when a users shares a link to a channel via the YouTube app.
In this commit we add the missing "https://youtube.com/" option to the
"isYoutubeUrl" function.
It is now possible to use a custom Nitter instances. This means a user
must not rely on our Nitter instance and can instead use his own
instance. To use a custom Nitter instance a user must provide the full
RSS feed url for the instance.
If a RSS feed entry did not contain a "published" field, we skipped the
entry and didn't added it to the database. Now it is also possible to
use feeds without a "published" field. If a feed entry does not contain
the field it must contain a "updated" field, which we then use for the
"publishedAt" field in the database. If a feed entry doesn't contain one
of the two fields the entry is still skipped.
This commit improves the parsing of the RSS feeds for a podcast, because
some podcasts were not working as expected:
- The "https://gotime.fm/rss" podcast uses the "itunes:image" field for
the podcast icon instead of "image", so that we now looking at both
fields to add the podcast icon.
- The "https://itsallwidgets.com/podcast/feed" podcast didn't contain a
link in the entries, so that all entries were skipped. For that a
podcast must not contain a link anymore to be parsed, but most contain
a media file which can be played within the app.
- Adjust "Release" section in contributing guide.
- Disable "X" datasource, since it is not working after the latest API
adjustments.
- Update "version" key and "msix_config.msix_version" key in
"pubspec.yaml" file.
- Change user in "AddSourceReddit" widget help text.
- Enable macOS App Sandbox.
- Adjust logo for Windows version.
Improve the media handling within the app. We do not save the media
files for items to the Supabase storage anymore. The source icons are
now only saved in the Supabase storage, the usage of an url as source
icon is not possible anymore.
The media files for items are directly retrieved from the corresponding
url or for the web version from the "image-proxy-v1" Supabase function.
If the image is retireved from the Supabase function we cache the image
in the browser via the cache control headers. If the image is directly
retrieved from it's url it's cached by the "CachedNetworkImage" widget.
It is now possible to use a Nitter instance, where the RSS endpoints are
protected via basic authentication. For that a new
"FEEDDECK_SOURCE_NITTER_BASIC_AUTH" environment variable was added,
which can be used to set the value for the authentication header.
