STORM UPDATE: Daily Bail To Be Shut Down Later Today
Sandy rocks the Bail.
UPDATE 11 pm EST - The Squarespace team is keeping 1.4 million sites online by carrying cans of fuel up 17 flights to the roof to keep their generator operating.
It could happen at any minute. We have been told that Daily Bail and a million other websites will likely go dark sometime this afternoon. Our site platform and hosting provider, Squarespace, was hit hard last night by the floods in NYC.
We have no way of knowing how long the outage will last (I'm guessing LESS than 24 hours), but since no one can shut down the financial insanity, not even a hurricane, we will continue posting links and stories via Twitter. So, if you don't follow us, now would be a good time to start.
Follow Daily Bail on Twitter:
https://twitter.com/TheDailyBail
---
Squarespace - Storm Update
GOOD NEWS
http://status.squarespace.com/
Update [6:25pm ET]
We have confirmed that the fuel pump is now working. We can now operate off of fuel indefinitely and there are continuous deliveries scheduled. Additional issues remain, the largest of which being that our generator will need to be taken offline at some point to clean the fuel filter. We can avoid that downtime if Peer1 can provide a street level generator. If this generator failure does occur, we expect the level of downtime to be on the order of an hour, instead of the multi-day outage we were facing at the onset of this crisis. We remain in a difficult situation, but the most difficult challenge we were presented with is now behind us. As of this writing, Squarespace has miraculously avoided all downtime related to Sandy, but we still have many challenges ahead with respect to getting back to anything normal.
Update [4:54pm ET]
A pump has been successfully installed that is carrying fuel to the roof fuel header, and it has been running for an hour. We are awaiting confirmation that the header and our tank are properly connected. The building is going to attempt to power up downed generators for other floors in 30 minutes.
Update [11:16am ET]
Some bad news.
We may not be able to deliver more fuel, as the building is shutdown mode. Last estimate is that we have about 3 hours left. Things change constantly, and we will keep you updated.
---
Everyone, I have some unfortunate news to share.
Our primary data center, Peer1, in Lower Manhattan lost power yesterday at about 4:30PM local time. At that time, we smoothly made the transition to generator power and took comfort over the fact that we had enough fuel to last three to four days. (Peer1 stayed online during the last 3 major natural disasters in the area, including a blackout that lasted for days.)
At 8:30PM yesterday, we received reports that the lobby in the data center’s building was beginning to take on water. By 10:30PM, as is sadly the case in most of Lower Manhattan, Peer1’s basement had experienced serious flooding. At 5AM, we learned our data center’s fuel pumps and fuel tanks were completely flooded and unable to deliver any more fuel. At 8AM, they reported that the generators would be able to run for a maximum of four more hours.
http://status.squarespace.com/
---
Update [10:38am ET]
Bucket brigade going strong. We’ve gone through half of our morning fuel delivery and are expecting a truck with 5,000 gallons coming at noon. Potential issues are lack of bucket brigade manpower into the night, and our fuel pumps burning out. Peer1 is working to establish a more permanent alternative to fuel delivery. If successful, it may be online this afternoon. Spirits are strong and everyone from Peer1, Fog Creek, and Squarespace is working together. We’ve added some photos below in response to requests.
Midnight Bucket Brigade
If we were there, we would offer a song and a beer.
Oct 30, 2012 at 1:43 PM
Reader Comments (27)
http://www.yummly.com/blog/wp-content/uploads/2012/05/Dutch-made-Dog-Beer.jpg
http://www.hlntv.com/slideshow/2012/10/24/animal-pet-cute-costume-halloween?hpt=hp_c2
Frightening being the operative word.
I fear that in the interim you will be replaced by an idealogically correct exact duplicate ...
I'll just kill myself now.
Why would we cover the krim case ?
A mentally incompetent nanny isn't really my cup of tea.
After that, reporters filed FOIA requests seeking documents relating to that fraud. In response, the S.E.C. produced documents only after redacting material information from them...
http://www.bloomberg.com/news/2012-10-28/why-does-the-sec-protect-banks-dirty-secrets-.html
It's worth noting that if capitalsm had been allowed to do its job by wiping Citigroup off the govt's balance sheet and into the history books, the redactions--based on the claim of trade secret information--would have been obviated, the concealed information would be in the public domain, and many of the perpetrators outed.
http://usabailout.com/content/1-2-3-citigroup’s-fraud-bailout-coverup-shows-real-purpose-bailouts
I thought that was their MO.
We have been running hour by hour by manually carrying fuel to our generators (17 floors) with support from the building. As the night goes on, this is becoming a bit more difficult to sustain, as fuel trucks will appear more intermittently. For now, we remain online. Thank you all for your patience.
As before, we do not have a sustainable solution to the energy problem, but we persevere in our efforts to get more fuel on site and get a pump connected.
http://status.squarespace.com/
http://usabailout.com/content/1-2-3-citigroup%E2%80%99s-fraud-bailout-coverup-shows-real-purpose-bailouts
It seems likely to me that this internal SEC decision was impacted by Robert Khuzami, a former Deutsche Bank executive with ties to Citi.
http://dailybail.com/home/busted-robert-khuzami-sec-enforcement-chief.html
Khuzami who once promised to be a tough cop on the beat, allegedly broke protocol, held a secret meeting with his good friend - counsel for Citi - then instructed SEC staff to go easy on Crittenden and Arthur Tildesley, the lying curbskank at the helm of Citi investor relations.
The U.S. Securities and Exchange Commission’s internal watchdog is reviewing an allegation that Robert Khuzami, the agency’s top enforcement official, gave preferential treatment to Citigroup Inc. executives in the agency’s $75 million settlement with the firm in July.
In the piece by William Cohan, Bowen says the SEC told him that they would make his documents available to reporters, so something changed, which means someone got involved to make that change. Hello Khuzami.
Update [1:02am ET]
We continue to manually provide fuel to the generator, a plan which appears to be working, but is likely not sustainable. The building now has powerful pumps clearing out the basement, which we hope will expose the main pump lines — which would allow us fuel for days. Hopefully our manual efforts, joined with the building, can see us through. We have loaded enough fuel to carry to the morning, at which point we’ll provide another status update.
http://www.quickmeme.com/meme/3rkl43/
---
They feel the daily bail love. Their work in the last 24 hours is the true definition of customer service.
http://status.squarespace.com/
They have photos up now showing their efforts.
Terrific progress this afternoon.
We have a few hundred gallons of spare fuel on the roof along with a full tank, which will take us well into the night. On top of teams from Peer1, Fog Creek, and Squarespace, we were able to hire additional help from Brooklyn and Queens. Special thanks to Mike Mazzei and his team at Peer1 for unprecedented dedication these past days.
We have a big crew teaming up tonight for a midnight bucket brigrade, and another in the morning. Traffic is terrible with gridlock everywhere and no public transport, so we are scheduling carefully.
We are cautiously optimistic that fuel will be running to the roof by end of day tomorrow or the day after. Water is being successfully pumped from the basement at a rate of one foot per hour. Fifteen feet of water remain.
We have increasing confidence that we may avoid downtime. Of course, there are no guarantees and we are working hour-by-hour.
Shut down might be coming in 30 minutes or so.
http://status.squarespace.com/post/34763425946/update-11-16am-et
The site is running so slowly today (and yesterday) because Squarespace is conserving power while they run on generator. Apparently they are using 400 gallons of diesel fuel every 2 hours or so.
Update [2:43pm ET]
We were able to temporarily resume carrying fuel for an hour, which means we have about 4-5 hours of fuel left on the roof.
Datacenter engineers are on site attempting to finish a more permanent pump solution, which we should know more about within the hour.
http://status.squarespace.com/image/34739054081
Midnight Bucket Brigade