As some of our members started handling bigger amounts of money in their PopAds accounts, we have decided to set up additional security systems.
At this moment, we have added activity logs, where all members can check their last login actions. Apart of that, we have added support for two factor authorization using YubiKey hardware token. All members can go to Security page under Account Manager and add existing YubiKey or order new one(act quickly, there is 20% discount coupon for our members!).

Prologue

The code that our publishers post on their websites loads our adcode that is responsible for opening PopAds popunders. In order to make it load as fast as possible, we use a Content Delivery Network.
It is a kind of hosting provider that hosts static content(like our Javascript adcode) on many servers around the globe, so each visitor downloads it from the closest location.
Another benefit of using a CDN is its distributed architecture, which should provide high reliability and stability.
In case the CDN stops serving our adcode, our system is unable to open popunders and effectively goes down.

Incident

Just few weeks ago, when we came up with the idea of full redundant adcode utilizing more than one Content Delivery Network, we did not expect it to be "tested in battle" so soon.
We thought it would be one of those many redundancy features that we have there just-in-case, but do not really expect them to be ever needed. The night between 29th and 30th May (GMT) showed that even highly distributed CDN networks with hundreds of Gbps bandwidth can stop working.
In the late evening on 29th May 2013, our automated monitoring system reported increased latency on our primary CDN as well as short downtimes every few minutes. After contacting the CDN provider, it has turned out that they are under a heavy DDOS attack and trying to mitigate it.
In other words, our primary CDN is not stable and there is nothing we can do, but wait for the provider to resolve the problem.





Not more than a month earlier, we somehow managed to forsee a situation like this and created a new, redundant publisher code that will download our adcode from secondary CDN if the primary is not reachable.
At the time of the incident, our new publisher code automatically switched to the backup CDN and the issues of our primary CDN did not cause any drop in traffic handled by PopAds.

Statistics

• Our primary CDN was completely down for 28 minutes,


• For more than 24 hours, the primary CDN was much slower than usually,


• More than 500 000 adcode requests were automatically and instantly rerouted to our backup CDN.

Recently we have update our adcode. We encourage all publishers to grab the new adcode using Code Generator and install it.
The new adcode brings following changes:

• Secondary adcode script location. The old adcode used only one Content Delivery Network(CDN). In case that one network experiences technical problems on one or more nodes, our adcode would stop working. New adcode resolves this problem by utilizing two separate CDNs.


• According to Cedexis, even best Content Delivery Networks drop as much as 0.5% of requests. Our new adcode resolves this problem by utilizing backup CDN.


• Default Minimal Bid has been set to 0. This means that publishers give us the control of their Minimal Bid so we can maximize their revenue. The reason behind this change is that we have plenty of traffic from countries like Vietnam and at the same time we have advertisers willing to buy it, but at lower rates. We believe it is better to sell a popunder at lower value then not to sell it at all. This change will not affect rates for countries for which we already have good rates. Also, publishers can always change it. Still, we advise to use default settings in the Code Generator for optimal revenue.

Yesterday we switched PopAds to our new backend system. The operation took approximately 3 hours, during which no major issues were faced. While at this moment, you might not see any difference except the panel is a bit faster, the new backend system gives us a lot of new options:

• All critical elements of the system are fully redundant. This means that we should be able to achieve very high uptime and system stability.


• Reports and dashboards will be supercharged, getting approx 3-4 times faster.


• New traffic registering engine will be able to forward popunders to advertisers landing page faster, resulting in improved conversions.


• Greatly increased capacity - with the new system we are able to handle up to approx. 100M popunders a day.


• Capacity can be easily and very quickly extended if needed, as the new backend is fully scalable.

In the new backend system, billing system works in a slightly different way. While reports are still realtime, billing is not. Campaign budgets and account balances are no longer updated realtime, but every minute.

In coming weeks we will be implementing many new features, including new powerful reporting engine. Stay tuned!

We are currently performing a database maintenance. PopAds might be working a bit slower during next few hours.
Also, advertisers might not be able to change/submit campaigns for approx. 15 minutes.

We apologize for the inconvenience.