We build. You grow.

Get best community software here

Start a social network, a fan-site, an education project with oxwall - free opensource community software

Is Oxwall dependable for 1 million community? | Forum

Ash
Ash Dec 10 '18

Hello All

Is oxwall having all the things needed to create and run a mega network? We are on seed stage and averagely funded and  exploring all options and i think we found below 3 as tested scripts for mega community sites

1)Dolphin

2)Oxwall

3)Elgg

Is oxwall dependable to launch a customized mega network like linkedin/facebook? What are other good options, 

feel free to comment your views, expertise and advice.


Thanks



AppXprt
AppXprt Dec 11 '18
Depends on multiple factors including the number of and specs of servers, bandwidth requirements, load balancing, DB and PHP performance, arrangement of resources, global server coverage, etc...


Probably could be scaled up on the right dedicated hardware on a great network with engineer's who know what they're doing.


There are issues like the cron script running every minute, the larger the site gets the longer it takes to run the cron script. If it exceeds 1 minute, the original script never finishes before the next one runs.


Solutions can be provided for most all issues though and if implemented correctly you should be able to scale most PHP scripts to any size necessary, but requires $$$$$$$ for hardware, software and Engineers / Developers. Honestly I recommend building on AWS because it makes it easier to scale multiple servers in multiple regions each under a load balancer for that region.


You also need Solution Architects / Solution Engineers, Programmers & Developers, Network & System Engineers / Admins, Site Admins & Moderators, Staff....


Need to design Secure Network Infrastructure including Separate Core and DB/Storage Networks and Servers, Secure Data Environment, Secure Domain, Firewalls, Routers, Reverse Caching Proxies, etc... 


The list goes on and on and on, because building a massive global network is much more than just running some cpanel web servers.


You wouldn't even want to use cpanel or WHM or ANY type of admin panel. This reduces performance, available resources and opens you up to security issues like exploits and vulnerabilities because of the larger attack surface, unless it is implemented correctly (only accessible over 2FA VPN behind Firewall.) 

The Forum post is edited by AppXprt Dec 11 '18
Senior Developer Leader
Senior Developer Dec 11 '18
I already tested Oxwall in my testing server with over 1 million testing accounts, I have it for testing my plugins. Quick answer: It works. 


I do not recommend Elgg.


Now where is the catch? All scripts you listed, they rely on ping requests. This means the more users connected simultaneously, the more powerful and faster servers you need.

To solve this you have 2 options: 


1 - Optimize the caching, optimize all plugins, add more hardware and distribute the load with more servers (Faster to do, cheaper in the short run).


2 - Create an extra plugin to make the server loads faster using sockets so the user doesn't need to smash the server with ping requests. This is something I have been working on the last year. This option is very expensive and needs plenty of work for any of this 3 Scripts (Dolphin, Oxwall, Elgg). This option is also cheaper in the long run (Years with millions of users). All big Social Networks switched from ajax to sockets and are saving lots of money in servers.



I suggest to use this software (Oxwall) in a powerful server and start growing, when you need more power, add more servers and start working on the option 2 to be ready to switch all your ajax calls to sockets. When you hit 10 million monthly users, it is time to make the switch so you can start saving money.

All I said is the way all big Social Networks have been doing.



Senior Developer.

The Forum post is edited by Senior Developer Dec 11 '18
Ash
Ash Dec 12 '18
Hi Sr Developer


Thanks for your kind response, so you mean to say is it works well on 1million level, I also understand that you have expressed limitations to all 3scripts i.e oxwall, dolphin and elgg.


1)It would be highly appreciated if you can recommend a good script? 


2)Many people recommend oxwall, but at the same time i see thats its not updated and the founder are away from engagement.. is there any longterm setbacks on this?


3)Basically our idea is to launch a social network where we would be at the same time allowing/helping our members to create their own networks.. something where they can convert their group via a sub-domain into a standalone buildable/scalable platform, (Here we are still figuring out how we will earn out of them? If you can suggest)


4)What should be a tentative cost i should consider to start with 10kmember capacity and promote it to acquire 1million within 2-3years..


look fw for a serious and concrete development on this,

Thanks & Regards

Ashish 
Quote from Senior Developer I already tested Oxwall in my testing server with over 1 million testing accounts, I have it for testing my plugins. Quick answer: It works. 


I do not recommend Elgg.


Now where is the catch? All scripts you listed, they rely on ping requests. This means the more users connected simultaneously, the more powerful and faster servers you need.

To solve this you have 2 options: 


1 - Optimize the caching, optimize all plugins, add more hardware and distribute the load with more servers (Faster to do, cheaper in the short run).


2 - Create an extra plugin to make the server loads faster using sockets so the user doesn't need to smash the server with ping requests. This is something I have been working on the last year. This option is very expensive and needs plenty of work for any of this 3 Scripts (Dolphin, Oxwall, Elgg). This option is also cheaper in the long run (Years with millions of users). All big Social Networks switched from ajax to sockets and are saving lots of money in servers.



I suggest to use this software (Oxwall) in a powerful server and start growing, when you need more power, add more servers and start working on the option 2 to be ready to switch all your ajax calls to sockets. When you hit 10 million monthly users, it is time to make the switch so you can start saving money.

All I said is the way all big Social Networks have been doing.



Senior Developer.


Ash
Ash Dec 12 '18
Hi ViXiV


Thanks a lot for your response, So i understand from your response is that we should focus more on hardware quality, security and alerts/notifications.. That's a great advice!


Can you also please help make me understand what are the possible points i should keep in mind when considering your inputs, keeping in mind below points


Hi Sr Developer
Thanks for your kind response, so you mean to say is it works well on 1million level, I also understand that you have expressed limitations to all 3scripts i.e oxwall, dolphin and elgg.
1)Many people recommend oxwall, but at the same time i see that, its not updated and the core team are away from engagement.. is there any long-term setbacks on this?
2)Basically our idea is to launch a social network where we would be at the same time allowing/helping our members to create their own networks.. something where they can convert their group via a sub-domain into a standalone buildable/scalable platform, (Here we are still figuring out how we will earn out of them? If you can suggest)
3)What should be a tentative cost i should consider to start with 10k member capacity and promote it to acquire 1million within 2-3years..
look fw for a serious and concrete development on this,Thanks & RegardsAshish 
Quote from ViXiV Technologies Depends on multiple factors including the number of and specs of servers, bandwidth requirements, load balancing, DB and PHP performance, arrangement of resources, global server coverage, etc...


Probably could be scaled up on the right dedicated hardware on a great network with engineer's who know what they're doing.


There are issues like the cron script running every minute, the larger the site gets the longer it takes to run the cron script. If it exceeds 1 minute, the original script never finishes before the next one runs.


Solutions can be provided for most all issues though and if implemented correctly you should be able to scale most PHP scripts to any size necessary, but requires $$$$$$$ for hardware, software and Engineers / Developers. Honestly I recommend building on AWS because it makes it easier to scale multiple servers in multiple regions each under a load balancer for that region.


You also need Solution Architects / Solution Engineers, Programmers & Developers, Network & System Engineers / Admins, Site Admins & Moderators, Staff....


Need to design Secure Network Infrastructure including Separate Core and DB/Storage Networks and Servers, Secure Data Environment, Secure Domain, Firewalls, Routers, Reverse Caching Proxies, etc... 


The list goes on and on and on, because building a massive global network is much more than just running some cpanel web servers.


You wouldn't even want to use cpanel or WHM or ANY type of admin panel. This reduces performance, available resources and opens you up to security issues like exploits and vulnerabilities because of the larger attack surface, unless it is implemented correctly (only accessible over 2FA VPN behind Firewall.) 


AppXprt
AppXprt Dec 12 '18
Great point by Senior Developer about the constant ICMP traffic (PINGS) from every client, this can basically cause the same issues associated with DDoS attacks. Too much traffic to any server will choke it and technically the network pipe only has to hit 50% bandwidth utilization before consistent failure occurs.


This definitely needs to be addressed as it significantly affects bandwidth consumption and server requests.

Like he has said, this is also used by multiple plugins for instance the Messaging system has it's own set of ICMP ping's it sends on top of the main ones from the OxWall system, compounding the issue for each plugin using ICMP.

The Forum post is edited by AppXprt Dec 12 '18
OW-Ghost
OW-Ghost Dec 13 '18

Hi Senior Developer


I was read your conversations 


When do you think this plugin will bee finished?


"Create an extra plugin to make the server loads faster using sockets so the user doesn't need to smash the server with ping requests"


I feel very interesting if it can save server cost, but i think very hard to make such plugin working with oxwall fully


ViXiV Technologies


Maybe you could create such plugin? You have server skills i think


Ow-ghost

The Forum post is edited by OW-Ghost Dec 13 '18
AppXprt
AppXprt Dec 14 '18
It's possible, but honestly this is a huge undertaking...


I would be willing to collaborate on this, but I don't see doing it alone lol

OW-Ghost
OW-Ghost Dec 14 '18
Hmmmm i wounder how much SD have left on this sockets plugin? Will you ever finish the project or do you need help finish it?


I was read more about sockets on this link https://stackoverflow.com/...ket-io-pros-and-cons


And wow it looks like sockets is much better then ajax....but how hard work is this? 


Again ,if it saves cpu and ram then you save money alot thats why i feel this very interesting 

The Forum post is edited by OW-Ghost Dec 14 '18
AppXprt
AppXprt Dec 14 '18
Websockets need to be built into Oxwall directly to replace the AJAX and ICMP utilization.
The Forum post is edited by AppXprt Dec 14 '18
OW-Ghost
OW-Ghost Dec 15 '18
Yes i think so to, alot of hard work almost like rewrite the oxwall core and plugins?


But if it saves money and work much better then AJAX then maybe the hard work would give good result in the end

JB TECH
JB TECH Dec 15 '18
To be completey honest, looking over the structure of Oxwall it seems as if it was intentionally meant to provide for smaller communit setups (say between 2,000 - 20,000). One of the biggest issues with the software and server use definitely comes from the initialization. If you have a lot of plugins, they all get initialised on load, so if you have X many users loading in and you add more ping requests from plugins like Messages, you have a mess on your hands.

I believe finding a way to re-engineer the init system can improve load times drastically, and perhaps find a way to use separate servers for media files (such as avatars, pluginfiles, etc.)
OW-Ghost
OW-Ghost Dec 15 '18
Someone should try remake one feature at the time. 


start with the plugin features that takes huge of load like the "messages" plugin.


And then jump on next heavy load feature. I think if take everything at one time it to much work.


And i think a plugin that can save a lot of "money" for all oxwall members would bee a best seller in store no doubt.


But as always it would bee hard make it work with skadate hybrid apps and skadates own plugins.


Ow-ghost

The Forum post is edited by OW-Ghost Dec 15 '18
Senior Developer Leader
Senior Developer Dec 15 '18
Jake - JB TECH Oxwall Support I strongly disagree, all CMS made on PHP  have the same way to load their plugins, the plugin developer is the one who must know how to optimize their plugins. It doesn't matter how fast the core is if plugin makers do not optimize their plugins. Oxwall was made to support as much users as your server can. If you optimize everything and make a few fixes here and there, I see no problem in having 1 million users with a few fast and strong servers.


OW-Ghost

The plugins that make the server slow are Messages/Chat/Mailbox and Newsfeed. This 2 plugins are in my TODO list for the first quarter of 2019, I will make new ones and I want to start using sockets on both. It will not work on shared hosting, it will work on dedicated servers and on VPS  because it needs software that shared hosting does not have.


Senior Developer.

JB TECH
JB TECH Dec 16 '18

I understand that, SD. Using the proper class and file structure for your plugins ensures better optimization, but there have been several instances and reports on this forum in the past of users who's websites (some even less then 500 members) being slow on their shared hosting due to having a large amount of plugins actively running. Each request initializes, and add on top of that if the plugin is poorly written, especially if it's manifesting another class of the script, it makes it worse.


When you think of a core demographic, I don't think too many people are running with VPS and dedicated servers, or dropping thousands a year to host their site. Any CMS PHP script will run fine even with hundreds of thousands of users on even mid-tier web hosting. But people need to take in account multiple factors, not just the base numbers of X users.


We have to look at the server limits as a variable, X amount of user accounts, X amount of average active users (usually by the daily or weekly), X amount of active plugins -> X amount of initializations made per request, and then you add plugins and functions that handle their own realtime server use such as mailbox and others becoming another variable in the equation.


The software should aim to cater to the majority, which are people trying to operate smooth communities averaging (I'd estimate) about 5k to 20k users, with maybe 40%-65% of those weekly actives. Yes, 1 million users would work on a "faster server," but what about 1 million active of 1 million on a install that has say 80 plugins all initializing per request? It's a big problem requiring more then one solution.

OW-Ghost
OW-Ghost Dec 16 '18
great news!


I would like see sockets working with this 2 plugins.


Specially the Messages/Chat/Mailbox plugin that many members is use more then the newsfeed. But again i know skadate have patched this plugin but maybe can work with skadate patch to. i think possible.


Ow-Ghost

The Forum post is edited by OW-Ghost Dec 16 '18
JB TECH
JB TECH Dec 16 '18

In short, you can make any data-based script you could dream of and have it run with any number of users provided the hardware is there.


But you can't just optimize the add-ons of said script and not the script itself. If a car's engine gets bogged down, you can't just purchase better tires and suddenly it's back to speed.

Senior Developer Leader
Senior Developer Dec 16 '18
Jake - JB TECH Oxwall Support I get your point, but I know that this Software can handle 1 million monthly users, I did some stress tests and did the math before starting making plugins for Oxwall. That's why always say Oxwall is good enough. 


I agree there is some things can be optimized, there is no perfect software. I did compare Oxwall with other software in the past and this was the best for us, in fact if you run stress tests and compare your results against  Wordpress, you will find out that Oxwall is one of the best options.
If you aim for 1 million users, you can't even think on shared hosting, no point in talking about it, shared is for little websites.

You say it needs more than a solution and I agree, that's exactly what ViXiV Technologies and I said:

If plugins get optimized, add a CDN, another server for the database and cache plugins there would be no issue at all for him to make his website for 1 million monthly users.



Ashish

Here are some answers:

1) I recommend Oxwall. 


2) Oxwall is the core code of Skadate too, as long as Skalfa still make money with skadate, Oxwall will still exists and be updated in the github repository. I see no problem in the future right now.


3) That's a lot of work, you will have to modify the core or make a new plugin for it. It can be done in a plugin but it needs very capable developers to make it without having to rewrite the core code in case Oxwall is updated by the Oxwall Team. The best way (I think) is taking the groups plugin, and rebuild it into a new plugin for your needs, just point all subdomains to this plugin and you are done. 

How will you earn? 

-Monthly subscriptions limiting the number of users that his group can have and features of this group.

-User credits is one of the best ways to monetize. 

-Run paid ads, just like Facebook does, they are the number 1 in the world and they make money running ads.

-Selling something.

-More ideas? Take a look at the monetization plugins.


4) This one I don't have the answer, you should hire an expert to run the stress tests required to know what servers you will need for starting and which hosting provider and do the math for you.



Senior Developer.

The Forum post is edited by Senior Developer Dec 16 '18
Ash
Ash Dec 27 '18
Thanks Senior Developer  with the clarity, Also i thank OW-Ghost  and ViXiV Technologies  for the insights..


Over all What i may make-out is:-

Oxwall is a great scrip to start and manage small niche communities - off course if one has tech expertise, much of things can be enhanced! But launching large scale community is a Big Risk. (On the contrary i am seeing elgg, which has higher modification capabilities and seems more flexible, open and free) What do you have to say about that? they are also soon coming with v3.0


My take as Business/Product innovator:-


Rather then fighting for the market share with big sharks, its better to decentralize traffic on multiple servers! Now as tech experts.. You all have to push your grey cells and lets take the innovation of open source to next level! 


Below can be the starting points to take the discussion further on more collaborative approach:- 

1)We need to make groups more stronger and make it sound more independent (eg: lets say groups = min.networks, like 2k-10k level communities..


2)Further to make the huge tech/plugin exercise less complex and to make it light, fast and effective.. [please pour your ideas ] One Though can be "We need to distribute the mini.networks(groups) load to various sub-domains on multiple servers.. something like independent multiple oxwall "mini.networks" scripts  owned and accountable by group owners.." 


3)If we are able to decentralise and connect multiple scripts running on various servers then we will need to give it an institutional look, So we will need to keep sign-in/sign-up common from all sub-domains..


4)Lastly people are smarter then we think, so everyone would wish to have their own brands/domain, so here we will need to figure out something with "web services" "bridging" etc.. to keep things in control from security and core-branding perspective.


Lastly, 

For revenue generation, if we can achieve all above, i am sure we will figure it out later.. but frankly My thought is to make it almost free for the people of world!



[If anyone would please like to contribute and join in this project aimed with a mission called "We rise, by lifting others" with a time line to complete by end of yr-2019, request to kindly message me what can be your individual contributions done? with a tentative cost as well.. if you can also please provide few links of active communities you are running it would be easy to evaluate (i used the word "evaluate" with due respect, dont mind because after-all it's going to be a matter of shelling some good amount of bugs)


Also - If anyone would like to join with co-funding, door as open! 


Look forward

Thanks & Regards

Ashish Tiwari

OW-Ghost
OW-Ghost Dec 27 '18
<<<<another server for the database<<<<


- Is this complicated or easy process to do for a totally amatuer?


- A guide would maybe be handy for oxwall software doing like this.


Last 2 questions:


How much cpu and ram would i save?


How much better speed would my website and apps get?

The Forum post is edited by OW-Ghost Dec 27 '18
Pages: 1 2 »