Notice the use of first here – it finds the first post with the correct feed item url. Why the first one? Shouldn’t it be the LAST one at least? And shouldn’t the list be sorted (say order(:post_id)) before taking first or last so it is at least predictable?
Since the post custom fields are never deleted together with the post, this almost means that the deleted first post will be found every time. Mostly like meaning that try(:post) will return nil and then a new post gets created EVERY TIME the RSS feed is scanned.
This, I believe, is the reason behind all those duplicated posts.
Shouldn’t the correct behavior be to:
First sort the posts by post_id (same as by date)
Filter out all the deleted posts
Take the last one
I would have made a PR if I know how to… but I’m not too comfortable with all those ActiveRecord stuff.
It is not just @kuyashi, I am also getting duplicate posts every single time that the RSS feed is scanned. If it is scanned three times, then three duplicated posts.
def get_existing_post
return PostCustomField.where(name: "autobot_source_url", value: source_url).first.try(:post)
end
Now, assume there are multiple records in post_custom_fields with the same autobot_source_url, all but the last one of them point to posts that are deleted.
I am not sure if PostCustomField.where(name: "autobot_source_url", value: source_url) will also return all records (included those pointing to deleted posts) or just the live one.
If it returns only the live one, then you’re probably correct that first == last. However, I’m not so sure about that assumption. AFAICT, PostCustomField will return all the records matching the autobot_source_url, including those pointing to deleted posts.
So, when you take the first of that, and access the :post, I’m not sure what will result. My suspicion is that it returns nil, so that the clause falls into post_creator.create! and a new post is created. Every single time, because you’ll always find the deleted post first.
@schungx , I have the exact same issue. Have been following along on this thread, thought i might have an older version, or maybe an issue with the feed and it’s not the case. Every refresh = duplicate post created.
@Jon_Hawkins, make sure you pull the latest one. It should solve a few problems.
On the duplicated posts, how many posts are duplicated each scan of the feed? I find that only posts that were ever deleted before will continue to add new duplicated posts indefinitely. However posts that were never deleted are never duplicated.
Are any of your feed posts deleted?
A good way to find out is to:
Open Data Explorer
select * from post_custom_fields where name='autobot_source_url' and value='your duplicated feed item URL here'
See if how many posts got returned
If > 1, then you have some deleted posts and that’s creating the duplication problem
I think i didi a rebuild of the app 4 days ago and have the latest build. This is my first plugin though, is there anything specific i need to do besides ./launcher rebuild app?
Im using zapier to build a custom RSS feed of only 1 item. When autobot is live on that zapier feed it will create the same post over and over again, as each interval is hit. Let me rebuild the app now and just confirm I have the latest greatest.
I’m fairly certain I’ve only deleted the first post created by that feed after noticing a duplicate but let me test again and let you know.
I have never changed the feed URL to another feed. I just deleted the entire record and created a new one from scratch. I think I went to edit the existing feed one time and noticed something weird when saving so figured I’d just remove it and start fresh.
If you deleted the first post, then you’ll get duplicated posts forever. You have to change the URL.
Basically the first post is the most important. If it is deleted, a new post will be created during every refresh.
Deleting the campaign won’t help because the plugin won’t distinguish between different campaigns. This is probably also a potential problem, if an item is ever shared between two different feeds in two different topics/categories. They’ll never duplicate in this case.
Sorry currently I don’t have much time to look into this plugin code. I will check it all and will do the improvements as soon as possible. May be in next couple of weeks.
Hey, I fixed this problem now. @schungx is correct. Deleted posts are causing the issue. But using the last method not resolving it (anyway as per his wish I added it too ). Instead I used unscoped option to include soft deleted posts while finding records. Anyway thanks for Stephan .
. I merged it. It seemed good to me and I didn’t tested well. Let me know if any issues raised.
). Instead I used