Topic subject says it all...
Before I randomly dig into the code of the photos plugin, does anyone have any clue of "where" should I start looking? Or maybe have a fix for it?
It is strange to believe that no one has yet reported such error...
We build. You grow.
Get best community software hereStart a social network, a fan-site, an education project with oxwall - free opensource community software
Deleted photos are NOT excluded from Newsfeed, resulting in a broken link [Answered] | Forum
Leader Michael
Jun 24 '11
I would wait until the advanced photo plugin bundled with v1.2 release is out in case they have already addressed this...
Julio Marchi
Jun 24 '11
If there was any clue of "when" the 1.2 version will be out, I would say YES. However, no one guarantees that it will happen in a few days, weeks or months.
If, at least, there was an "open Beta" or a "preview release candidate" for us to test and review it, I'd agree with you... :) But, for now, the only we can do if fix such minor issues until a major update is released.
I am still trying to figure out the plugins structure and code composition. If anyone has any clue about this issue, would be great to have it resolved. At least, it is a good learning tool. And, more we know, better we can contribute. ;)
If, at least, there was an "open Beta" or a "preview release candidate" for us to test and review it, I'd agree with you... :) But, for now, the only we can do if fix such minor issues until a major update is released.
I am still trying to figure out the plugins structure and code composition. If anyone has any clue about this issue, would be great to have it resolved. At least, it is a good learning tool. And, more we know, better we can contribute. ;)
Team Den
Jun 27 '11
Our dev team also encountered with the same problem. The main issue is that this list of photos is added as html code into newsfeed stream, And it is necessary to parse this item - it takes too much system resources. That's why we decided thus far to leave it as is.
Leader Michael
Jun 27 '11
I would recommend a closed beta. There will be people who would grab it and use then complain loads why bits dont work.
Unfortunately there are 2 sides here, both right. It can look untidy if left unresolved but... I would rather the more serious issues were fixed and tested. (IE9 login problem for now, thanks for the hack BTW)
OK I seem to have said loads and not come support one side or the other! (I'm good at that!)
Unfortunately there are 2 sides here, both right. It can look untidy if left unresolved but... I would rather the more serious issues were fixed and tested. (IE9 login problem for now, thanks for the hack BTW)
OK I seem to have said loads and not come support one side or the other! (I'm good at that!)
Adrian
Jun 27 '11
Good words Michael - is the "sitting on the fence" an Irish thing - sorry couldn't resist that.
I think a closed Beta would be good - with people involved who know php, code and have a busy site - then we could (a) fix on the fly and report back and (b) give the code a good work out.
Thoughts anyone?
I think a closed Beta would be good - with people involved who know php, code and have a busy site - then we could (a) fix on the fly and report back and (b) give the code a good work out.
Thoughts anyone?
Leader Michael
Jun 27 '11
An Irish thing. Nah, Just me being indecisive... or not. I cant decide.
Others may call it lazy.
If this update is going to be as impressive as they say I would rather it was delayed so they are sure it is right. Last thing neeed is a potential buggy update and it putting off new clients. Oxwall needs to be popular otherwise time spent in it will be lost and existing oxwall sites will die!
I do have faith in them though, They have came through for me in even really drastic times in beta days!
Others may call it lazy.
If this update is going to be as impressive as they say I would rather it was delayed so they are sure it is right. Last thing neeed is a potential buggy update and it putting off new clients. Oxwall needs to be popular otherwise time spent in it will be lost and existing oxwall sites will die!
I do have faith in them though, They have came through for me in even really drastic times in beta days!
Julio Marchi
Jun 28 '11
Guys,
In regards to the OxWall 1.2 RC, I agree with a "closed beta". No one wants a beta versions freely flowing and mistakenly being used by unadvised people, creating unnecessary support requests and spreading bug reports about what is in fact a WIP.
However, you guys are missing the point of this thread: the "error" regards to deleted items (not only photos) that are kept as broken links in newsfeeds.
I confess that I was beyond surprised after reading the Addenster answer. He made me feel like if, somehow, the newsfeed was coded all wrong. That's why, before answering his message, I've decided to take a closer look into the source code. This way I could, at least, have the decency to reply him with a potential new idea for (maybe) a usable solution.
As far as I can see, the Newsfeed compile its data once and then save the resultant HTML in a static cache for future usage. It is NOT the "smarty" cache, it is a local specific static HTML, which seems to be created PER USER and not PER SYSTEM. This way, on sites with huge databases and tons of users, the newsfeed won't overload the server and will always perform fast, rebuilding only whatever is new, and attaching it to the specific static cached HTML.
As intelligent this solution might seem, the implementation was not done with the thoughts of "newsfeeds changes" in mind. I believe it is also related to the fact that photos and videos plugins were created after newsfeed plugin. Notwithstanding, the synchronization process requires too many resources, and if the page is rebuilt from scratch every time it is accessed, the server will eventually crash or (if you are lucky enough) slowdown as result of overloading.
With this scenario ahead of me, my goal was to think of a potential solution that wouldn't require the plugins to be completely rewritten and, at same time, that could keep the high performance as its priority. Based on these requirements, here is my conceptual implementation for us to solve such limitations and gain more power:
The plugins should be modified in a way that each plugin keeps small chunks of newsfeeds HTML, organized by users, time and date, and with some internal flags to identify what kind of operation was performed (the chunk can be one data and the flags another, both in the same Row of the same Table in the DB). This way, the plugin itself will be able to modify any chunk of older newsfeed based on users further modifications. I.E.: if an user deletes a photo, the plugin can read from DB the specific chunk of pre-built HTML that refers to that operation, modify it to avoid a broken link and maybe replace the broken thumbnail for one that says "deleted photo", then save the data keeping its referenced date unaltered. After that, the plugin can create a new chunk of HTML for the current transaction (a photo deletion) and add it at the Newsfeed list for current date/time/user. Such operations can be highly customized, and it is not a real challenge to create such "plugin based" data management (as each plugin will handle its data as they best understand it). It is not different than what a LOG does, only more elaborated.
In regard to the Newsfeed plugin, it will be only required to obtain the pre-created chunks of data and build the page on-demand based on the current data and how many feeds are marked to be listed. It is even possible to add a real-time updates using AJAX without requiring users to refresh the page.
In another words: changes on newsfeed contents (plugin based) should be handled and pushed by the plugin itself. It will also allow newer plugins to add data to the Newsfeed without requiring any change on the newsfeed plugin (I don't know how it is done today).
In general, even if such operations add a little bit more processing requirements to the plugins, it shouldn't be noticeable. The performance will probably be good as it is right now, but with a lot more power and options (including the so much desired PRIVACY).
Some other advantages can be obtained with such model:
1) Newsfeed can be programmed to ignore items (per user config), letting users to better customize their pages and preferences;
2) Users should be able to select, in real-time, specific newsfeed types to view or hide (i.e.: only photos, or only videos, or maybe photos and videos together, or yet only profile updates or per use newfeeds, etc);
3) It can be filtered by user, masked by date or range of dates and can even handle external feeds (from Twitter, Facebook, etc).
4) Newsfeeds will be able to reference back to other newsfeed entries. I.e.: A user delete a photo and I want to know when he/she did it. The deleted photo thumbnail can be programmed to jump to the newsfeed that display details of the deletion transaction (for instance);
5) Users can be allowed to delete/hide his newsfeeds related to some dates/operations/plugins from the feed and it will reflects to everyone immediately;
6) Did I mention CUSTOM PRIVACY OPTIONS on the plugin level? :)
I can think of a lot other possibilities and advantages for a Newsfeed plugin when the other plugins actually handle the updated data, especially if we keep it into the DB and use the power of SQL Queries to do the jobs of filtering and combining data for us.
Notwithstanding, we also have the Smarty helping the entire site with a great caching level and the MySQL query caches that can be programmed/used to minimize the amount of simultaneous access to the DB.
Well... This is how I imagine it could be better done. I hope it helps a little bit. :)
Let me know what do you guys think of such implementation model.
In regards to the OxWall 1.2 RC, I agree with a "closed beta". No one wants a beta versions freely flowing and mistakenly being used by unadvised people, creating unnecessary support requests and spreading bug reports about what is in fact a WIP.
However, you guys are missing the point of this thread: the "error" regards to deleted items (not only photos) that are kept as broken links in newsfeeds.
I confess that I was beyond surprised after reading the Addenster answer. He made me feel like if, somehow, the newsfeed was coded all wrong. That's why, before answering his message, I've decided to take a closer look into the source code. This way I could, at least, have the decency to reply him with a potential new idea for (maybe) a usable solution.
As far as I can see, the Newsfeed compile its data once and then save the resultant HTML in a static cache for future usage. It is NOT the "smarty" cache, it is a local specific static HTML, which seems to be created PER USER and not PER SYSTEM. This way, on sites with huge databases and tons of users, the newsfeed won't overload the server and will always perform fast, rebuilding only whatever is new, and attaching it to the specific static cached HTML.
As intelligent this solution might seem, the implementation was not done with the thoughts of "newsfeeds changes" in mind. I believe it is also related to the fact that photos and videos plugins were created after newsfeed plugin. Notwithstanding, the synchronization process requires too many resources, and if the page is rebuilt from scratch every time it is accessed, the server will eventually crash or (if you are lucky enough) slowdown as result of overloading.
With this scenario ahead of me, my goal was to think of a potential solution that wouldn't require the plugins to be completely rewritten and, at same time, that could keep the high performance as its priority. Based on these requirements, here is my conceptual implementation for us to solve such limitations and gain more power:
The plugins should be modified in a way that each plugin keeps small chunks of newsfeeds HTML, organized by users, time and date, and with some internal flags to identify what kind of operation was performed (the chunk can be one data and the flags another, both in the same Row of the same Table in the DB). This way, the plugin itself will be able to modify any chunk of older newsfeed based on users further modifications. I.E.: if an user deletes a photo, the plugin can read from DB the specific chunk of pre-built HTML that refers to that operation, modify it to avoid a broken link and maybe replace the broken thumbnail for one that says "deleted photo", then save the data keeping its referenced date unaltered. After that, the plugin can create a new chunk of HTML for the current transaction (a photo deletion) and add it at the Newsfeed list for current date/time/user. Such operations can be highly customized, and it is not a real challenge to create such "plugin based" data management (as each plugin will handle its data as they best understand it). It is not different than what a LOG does, only more elaborated.
In regard to the Newsfeed plugin, it will be only required to obtain the pre-created chunks of data and build the page on-demand based on the current data and how many feeds are marked to be listed. It is even possible to add a real-time updates using AJAX without requiring users to refresh the page.
In another words: changes on newsfeed contents (plugin based) should be handled and pushed by the plugin itself. It will also allow newer plugins to add data to the Newsfeed without requiring any change on the newsfeed plugin (I don't know how it is done today).
In general, even if such operations add a little bit more processing requirements to the plugins, it shouldn't be noticeable. The performance will probably be good as it is right now, but with a lot more power and options (including the so much desired PRIVACY).
Some other advantages can be obtained with such model:
1) Newsfeed can be programmed to ignore items (per user config), letting users to better customize their pages and preferences;
2) Users should be able to select, in real-time, specific newsfeed types to view or hide (i.e.: only photos, or only videos, or maybe photos and videos together, or yet only profile updates or per use newfeeds, etc);
3) It can be filtered by user, masked by date or range of dates and can even handle external feeds (from Twitter, Facebook, etc).
4) Newsfeeds will be able to reference back to other newsfeed entries. I.e.: A user delete a photo and I want to know when he/she did it. The deleted photo thumbnail can be programmed to jump to the newsfeed that display details of the deletion transaction (for instance);
5) Users can be allowed to delete/hide his newsfeeds related to some dates/operations/plugins from the feed and it will reflects to everyone immediately;
6) Did I mention CUSTOM PRIVACY OPTIONS on the plugin level? :)
I can think of a lot other possibilities and advantages for a Newsfeed plugin when the other plugins actually handle the updated data, especially if we keep it into the DB and use the power of SQL Queries to do the jobs of filtering and combining data for us.
Notwithstanding, we also have the Smarty helping the entire site with a great caching level and the MySQL query caches that can be programmed/used to minimize the amount of simultaneous access to the DB.
Well... This is how I imagine it could be better done. I hope it helps a little bit. :)
Let me know what do you guys think of such implementation model.
The Forum post is edited by Julio Marchi Jun 28 '11
Team Den
Jun 28 '11
@Julio
Thanks for detailed explanation. You are right, newsfeed stores all it's data into a database. And every entry of newsfeed's activity already has a chunk/flag/timestamp values :) When you upload new photo and after a few days somebody will leave a comment on your photo, newsfeed will raise up your entry about new uploaded photo with latest comment. System will not just create new entire about new comment, it will added this comment to existing entry. Other plugins work in the same way :)
The main problem is that when user uploads more then 1 one photo at one time, newsfeed saves in entry's chunk the whole html code for all uploaded photos, something like this:
"
, 
, 
"
and when user deletes photo 2, it is not so easy to removefrom chunk. It can be done by parsing chunk, but it too heavy for system resources.
In any way, we consider how to solve it as safely as possible for server's resources.
Newsfeed in oxwall is not a simple activity stream, it is something more powerful. Currently, all plugins can add and modify it's own actions via newsfeed API :)
Also, with new update, there will be added points 1,2 and 6 of your proposal list. Point 5 already exists in current version, isn't it?
Thanks for detailed explanation. You are right, newsfeed stores all it's data into a database. And every entry of newsfeed's activity already has a chunk/flag/timestamp values :) When you upload new photo and after a few days somebody will leave a comment on your photo, newsfeed will raise up your entry about new uploaded photo with latest comment. System will not just create new entire about new comment, it will added this comment to existing entry. Other plugins work in the same way :)
The main problem is that when user uploads more then 1 one photo at one time, newsfeed saves in entry's chunk the whole html code for all uploaded photos, something like this:
"
, , " and when user deletes photo 2, it is not so easy to remove
from chunk. It can be done by parsing chunk, but it too heavy for system resources.In any way, we consider how to solve it as safely as possible for server's resources.
Newsfeed in oxwall is not a simple activity stream, it is something more powerful. Currently, all plugins can add and modify it's own actions via newsfeed API :)
Also, with new update, there will be added points 1,2 and 6 of your proposal list. Point 5 already exists in current version, isn't it?
Julio Marchi
Jun 28 '11
Addenster,
It is not that hard to parse HTML using PHP, especially when the HTML is auto-generated (and so you can add "edition mark points" to be used as identifiers).
Another solution, actually a better solution, is to save those chunks in XML format. PHP has an internal parser for it and to convert XML to HTML is a breeze. PHP won't feel a thing and it can be done on-the-fly! ;)
Any word on the 1.2 version? I have lots of ideas, fixes and implementations but before waste my time with the 1.1.1 version, I'd like to have the feeling of what has been addressed in the 1.2 version and what will be actually worth to invest my time on.
How could I be of any assistance?
It is not that hard to parse HTML using PHP, especially when the HTML is auto-generated (and so you can add "edition mark points" to be used as identifiers).
Another solution, actually a better solution, is to save those chunks in XML format. PHP has an internal parser for it and to convert XML to HTML is a breeze. PHP won't feel a thing and it can be done on-the-fly! ;)
Any word on the 1.2 version? I have lots of ideas, fixes and implementations but before waste my time with the 1.1.1 version, I'd like to have the feeling of what has been addressed in the 1.2 version and what will be actually worth to invest my time on.
How could I be of any assistance?
Team Den
Jun 29 '11
We are working on 1.2. All I can say: it is will be available as it will be ready :)
Just be an active member of community. At this stage it will be the best way to help!
BTW Do you plan to create any plugin or theme soon for oxwall?
How could I be of any assistance?
Just be an active member of community. At this stage it will be the best way to help!
BTW Do you plan to create any plugin or theme soon for oxwall?
Julio Marchi
Jun 29 '11
Yes, I do have plans... I am already working on some Themes ideas (it is a breeze in OxWall, but sadly it lacks in regards to the login screen and other few screens that require hard coding. You should think of it in the 1.2 version).
For the plugins I have several ideas too, but I am only "learning" and "studding" about OxWall ways at this point. One thing that is slowing me down is the 1.2 version. As it is to be a huge advance in regards to its previous versions, I am holding my horses with my ideas (because I don't know how new changes will affect old plugins/themes, or even what new features will be present).
That's why I keep asking for a Closed Beta or a Release Candidate. With it in my hands I can start learning the 1.2 features and capabilities to bring some of my ideas to life ASAP. I believe I also speak in behalf of many others when I say this!
You should take it in serious consideration. We are here to have out hands dirty too! ;)
For the plugins I have several ideas too, but I am only "learning" and "studding" about OxWall ways at this point. One thing that is slowing me down is the 1.2 version. As it is to be a huge advance in regards to its previous versions, I am holding my horses with my ideas (because I don't know how new changes will affect old plugins/themes, or even what new features will be present).
That's why I keep asking for a Closed Beta or a Release Candidate. With it in my hands I can start learning the 1.2 features and capabilities to bring some of my ideas to life ASAP. I believe I also speak in behalf of many others when I say this!
You should take it in serious consideration. We are here to have out hands dirty too! ;)
Leader Michael
Jun 29 '11
whatever is decided, closed beta or not, good luck.
Julio. I do hope you are here for the long haul. You seem to be the most enthuastic and capable user to date. A valuable asset to the community!
(I'm still in love with you for the IE9 fix) LOL!
Julio. I do hope you are here for the long haul. You seem to be the most enthuastic and capable user to date. A valuable asset to the community!
(I'm still in love with you for the IE9 fix) LOL!
Julio Marchi
Jun 29 '11
I am here to stay... Unless I am kicked out! :)
Because of my line businesses and my technical background, I make usage of (or, somehow, I offer support to) several "systems": from WordPress to Joomla; from Sharetronix to Prestashop..., and I am very well educated about all of them. Based on my long term experience with so many different systems, I can only tell you how impressed I am with OxWall. Everything in the code make sense when you understand "why" things were done in such way, and besides of the glitches, everything it is very well done!
However, besides its solidness, the OxWall is yet a young system and there are lots of room for improvement, many "tweaks" pending in several aspects and uncountable new features yet to be added.
To make sure we can bring it closer to perfection, we need to discuss, explore and try new ideas. Besides (I am sorry for being this much honest) it doesn't seem smart to hold secretively a new release with many promises of improvements until it is ready to go. At least the active community should be able to try and explore it. It is good to identify bugs, to refine processes and to gather new ideas. It is also good to measure the general reaction in regards to the new features and etc...
Another side effect of new releases promises without "open betas" is what is happening with me: I have several ideas that I won't start executing over the 1.1.1 for obvious reasons. I will wait for the 1.2 version to be released before get my hands on anything. But, depending on how long it takes (and as we have no RC preview of it), who knows if I will still be this much interested in execute any of my ideas by the time the new version is released? So many things can happen when time passes... I am using myself as an example, but based on my experiences I can pretty much guarantee other potential developers will share similar feeling. Usually no one like to say anything like that out loud in the open, but I do because I care!!!
Finally, I am using OxWall in a personal project, and because of that I have all the best interest in having it as the best of the bests! I am willing to help if I am allowed to. I'll invest my time in developing what I need, and I will be glad to share the results with the community. But not before the 1.2 is available...
Meanwhile, I am studying the OxWall code and (whenever I can) helping others in the forum with my yet small knowledge about the system.
As I said: OxWall team can count on me!
Because of my line businesses and my technical background, I make usage of (or, somehow, I offer support to) several "systems": from WordPress to Joomla; from Sharetronix to Prestashop..., and I am very well educated about all of them. Based on my long term experience with so many different systems, I can only tell you how impressed I am with OxWall. Everything in the code make sense when you understand "why" things were done in such way, and besides of the glitches, everything it is very well done!
However, besides its solidness, the OxWall is yet a young system and there are lots of room for improvement, many "tweaks" pending in several aspects and uncountable new features yet to be added.
To make sure we can bring it closer to perfection, we need to discuss, explore and try new ideas. Besides (I am sorry for being this much honest) it doesn't seem smart to hold secretively a new release with many promises of improvements until it is ready to go. At least the active community should be able to try and explore it. It is good to identify bugs, to refine processes and to gather new ideas. It is also good to measure the general reaction in regards to the new features and etc...
Another side effect of new releases promises without "open betas" is what is happening with me: I have several ideas that I won't start executing over the 1.1.1 for obvious reasons. I will wait for the 1.2 version to be released before get my hands on anything. But, depending on how long it takes (and as we have no RC preview of it), who knows if I will still be this much interested in execute any of my ideas by the time the new version is released? So many things can happen when time passes... I am using myself as an example, but based on my experiences I can pretty much guarantee other potential developers will share similar feeling. Usually no one like to say anything like that out loud in the open, but I do because I care!!!
Finally, I am using OxWall in a personal project, and because of that I have all the best interest in having it as the best of the bests! I am willing to help if I am allowed to. I'll invest my time in developing what I need, and I will be glad to share the results with the community. But not before the 1.2 is available...
Meanwhile, I am studying the OxWall code and (whenever I can) helping others in the forum with my yet small knowledge about the system.
As I said: OxWall team can count on me!
The Forum post is edited by Julio Marchi Jun 29 '11
Leader Michael
Jun 30 '11
good then look at my faults and work out why my members are crying at me about links not working!!! :o)
Seriously, encouraging words! but develop away because I reckon there will be an Oxwall 1.3, 1.4 and even a 2.0!
Oxwall have released updates in the past only to be hit hard with a little error which had a few bad effects on the community installs. Not everyone will backup, test and install, they will jump in and sound surprised when it is broken.
In Oxwall's defense they did fix well and do have patience when it comes to people just not reading things!
Seriously, encouraging words! but develop away because I reckon there will be an Oxwall 1.3, 1.4 and even a 2.0!
Oxwall have released updates in the past only to be hit hard with a little error which had a few bad effects on the community installs. Not everyone will backup, test and install, they will jump in and sound surprised when it is broken.
In Oxwall's defense they did fix well and do have patience when it comes to people just not reading things!
Julio Marchi
Jun 30 '11
Michael,
That's why I mentioned "Closed Beta" (or a RC - Release Candidate). But, I don't want to extend this subject any longer. I already exposed my opinion and had the chance to defend my point. I don't think it is right to be using the forum to force my opinion or create any "flame". Regardless of the decision of how OxWall development team wants to work this out, I am here to help and I am willing to invest a decent amount of time on new developments in the future.
In regards to the photos broken link issue (the original subject of this thread), I've been looking into that and, at this point, it will require a lot of work to get it fixed. It probably doesn't worth the investment when we are on a verge of a new release that can potentially have it fixed or a different structure for those plugins.
Nevertheless, besides the broken links of deleted photos/videos, there is another problem: the translations. As the strings are also hard-coded in the local static files, when you apply the translations the strings in the news-feeds (dated before the translation is applied) are not translated on-the-fly, resulting in a Babel Tower news-feed type.
The solution for both cases is the same, but there is too much work involved to have it fixed. Also, it is presumable that a new version have more chances to reach shores before any work is done on those matters. I can only wonder about it, but I assume this new version will have most of those issues addressed, and IMO the better thing to do at this point is to wait and hope it doesn't take too long for the 1.2 version see daylight... :)
That's why I mentioned "Closed Beta" (or a RC - Release Candidate). But, I don't want to extend this subject any longer. I already exposed my opinion and had the chance to defend my point. I don't think it is right to be using the forum to force my opinion or create any "flame". Regardless of the decision of how OxWall development team wants to work this out, I am here to help and I am willing to invest a decent amount of time on new developments in the future.
In regards to the photos broken link issue (the original subject of this thread), I've been looking into that and, at this point, it will require a lot of work to get it fixed. It probably doesn't worth the investment when we are on a verge of a new release that can potentially have it fixed or a different structure for those plugins.
Nevertheless, besides the broken links of deleted photos/videos, there is another problem: the translations. As the strings are also hard-coded in the local static files, when you apply the translations the strings in the news-feeds (dated before the translation is applied) are not translated on-the-fly, resulting in a Babel Tower news-feed type.
The solution for both cases is the same, but there is too much work involved to have it fixed. Also, it is presumable that a new version have more chances to reach shores before any work is done on those matters. I can only wonder about it, but I assume this new version will have most of those issues addressed, and IMO the better thing to do at this point is to wait and hope it doesn't take too long for the 1.2 version see daylight... :)
The Forum post is edited by Julio Marchi Jun 30 '11
Leader
Team
