denise: Image: Me, facing away from camera, on top of the Castel Sant'Angelo in Rome (Default)
Denise ([staff profile] denise) wrote2011-04-08 04:53 am

(no subject)

I've seen people wondering why imports are taking so @)#$(*& long. This might help shed some light:



#229690254 2011-04-08 08:13:11 [38 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229690299 2011-04-08 08:13:31 [38 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229691506 2011-04-08 08:17:12 [34 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229693287 2011-04-08 08:22:58 [29 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229693552 2011-04-08 08:23:24 [28 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229694558 2011-04-08 08:26:19 [25 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229696494 2011-04-08 08:30:26 [21 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229697174 2011-04-08 08:31:00 [21 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229699435 2011-04-08 08:37:17 [14 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229699747 2011-04-08 08:38:02 [14 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Entries
State: failed at least once, will retry very soon, priority undefined.

#229700502 2011-04-08 08:39:28 [12 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229701320 2011-04-08 08:41:23 [10 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Verify
State: failed at least once, will retry in Never, priority undefined.

#229704262 2011-04-08 08:43:44 [8 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.

#229705151 2011-04-08 08:44:30 [7 minutes ago] in DW::Worker::ContentImporter::LiveJournal::Comments
State: failed at least once, will retry very soon, priority undefined.


1) LJ is still under wicked load due to the DDoS they're experiencing. This can mean the site timing out when we try to contact them for the data.

2) Every time LJ is inaccessable when the importer tries to hit it, the job goes on pause to retry again in a few seconds. It retries up to five times before giving up.

3) There are lots and lots and lots of people trying to import right now. (This means that imports from sites other than LJ will be slow too, because they enter the queue behind all the LJ import jobs.)

If I had to guess, I'd say the existing import queue as it stands right this very minute would take at least 24 hours to clear, because of a combination of the three above items. Please, please, please be patient!
sharpest_asp: Black and white art of a hissing snake arounda  dagger (Default)

[personal profile] sharpest_asp 2011-04-08 01:59 pm (UTC)(link)
Passing the word along as best I can, and I saw numerous tweets to that effect too from other friends.

Thank you, again, for trying to keep us in the loop, and I think a lot of us will try and keep others there!
amadi: A bouquet of dark purple roses (Default)

[personal profile] amadi 2011-04-08 03:22 pm (UTC)(link)
Is it reasonable to say that right now is not the best time to attempt to import from LJ?

I understand that people want to protect their content, but with their servers under attack, importing right now is just adding to the strain, and clearly their servers cannot keep up.
amadi: A bouquet of dark purple roses (Default)

[personal profile] amadi 2011-04-09 01:00 am (UTC)(link)
Gah. I've been vocal in my disapproval of some things LJ has done, but this is unconscionable.
stormy: βͺ ππŽπ“πˆπ‚π„ ❫ 𝑫𝑢 𝑡𝑢𝑻 𝑻𝑨𝑲𝑬 𝑴𝒀 𝑰π‘ͺ𝑢𝑡𝑺 ⊘ (Default)

[personal profile] stormy 2011-04-08 03:47 pm (UTC)(link)
This reminds me that you mentioned you were willing to teach programming to help out Dreamwidth. I'm still interested in helping out with small projects. Perhaps you have a few links of interest that I can research with? I even braved the suggestion queue a few minutes ago to put a few more ideas forth. I also wouldn't mind being teamed up with a programmer to write the html/css end of their projects, too.
foxfirefey: A guy looking ridiculous by doing a fashionable posing with a mouse, slinging the cord over his shoulders. (geek)

[personal profile] foxfirefey 2011-04-08 05:29 pm (UTC)(link)
I think the best way to start with that is to drop by #dw_dev in IRC or make a post to [site community profile] dw_dev_training talking about what you want to do.
sophie: A cartoon-like representation of a girl standing on a hill, with brown hair, blue eyes, a flowery top, and blue skirt. β˜€ (Default)

[personal profile] sophie 2011-04-09 01:25 am (UTC)(link)
That's #dreamwidth-dev, I think :)
Edited 2011-04-09 01:26 (UTC)
foxfirefey: A cat with a fish bowl on its head. (space cadet)

[personal profile] foxfirefey 2011-04-09 01:26 am (UTC)(link)
Ack, you're right! Fortunately the IRC wiki page will have the right stuff.
foxfirefey: A guy looking ridiculous by doing a fashionable posing with a mouse, slinging the cord over his shoulders. (geek)

[personal profile] foxfirefey 2011-04-08 06:21 pm (UTC)(link)
Suggestion I just thought of: maybe put an entry like this in [site community profile] dw_maintenance? Or would that be a bad idea since it's not really maintenance on our part...
foxfirefey: A firework bursts over the Las Vegas night skyline. (yay)

[personal profile] foxfirefey 2011-04-09 12:54 am (UTC)(link)
Woo woo!
ariestess: (Default)

[personal profile] ariestess 2011-04-08 08:50 pm (UTC)(link)
I figured this sort of thing would be happening. I'm to the point where I'm not even retrying to crosspost from DW to LJ. If it fails the first time, I give it a good chunk of time before retrying. And I'm waiting for everything to be much more stable on the LJ side before helping my other half with doing an import/backup of her LJ again.
jecook: (Default)

[personal profile] jecook 2011-04-08 10:13 pm (UTC)(link)
Yep.

Hell, LJ's been flagging ME, a meatbag, as a bot. Not surprising that it's flagging the importer as a bot.

[personal profile] mashka_romashka 2011-04-08 11:17 pm (UTC)(link)
Thanks a lot. I've imported all my posts about my kids, it was very important for me
con_girl: (Default)

[personal profile] con_girl 2011-04-08 11:36 pm (UTC)(link)
This is what I get for not getting around to it (importing stuff over to DW). I hope things are going okay for DW generally. This must be a busy time.
andrewducker: (Default)

[personal profile] andrewducker 2011-04-09 02:24 pm (UTC)(link)
Is it worth having multiple queues, so that non-LJ imports are less affected?

Or, if an import from site X fails, switching to the next import from a different site, before going back to site X again?
andrewducker: (Default)

[personal profile] andrewducker 2011-04-10 05:23 am (UTC)(link)
Good point - that does make sense.
dragonchilde: Rear view of a woman in a pale renaissance-style dress (Default)

[personal profile] dragonchilde 2011-04-18 05:18 pm (UTC)(link)
I just wanted to say thank you for this feature; I know this is #oldpost and #threadzombieism here, but I wanted to thank you for the importer... i've had a DW journal for about a year and a half, but haven't done much with it. The recent issues at LJ had me worried for the loss of irreplaceable data... and with facebook stealing the majority of my active friends there away, I figured what the hey... I have little to lose.

And now, I've found that DW is my default. I LIKE it here better. It's cleaner, more stable, and truthfully... it's the ethics here that have really drawn me in. It feels like Old LJ.

I still love LJ, and I'll have an account there as long as they let me... but I think I'll make this my new home base. :)
dragonchilde: Rear view of a woman in a pale renaissance-style dress (Default)

[personal profile] dragonchilde 2011-04-18 09:45 pm (UTC)(link)
If you guys keep up the exemplary company philosophy and work ethic... I think you'll set a whole new standard for web 2.0 business.
azurelunatic: A glittery black pin badge with a blue holographic star in the middle. (Default)

[personal profile] azurelunatic 2011-04-19 03:25 pm (UTC)(link)
Is it about that time again? It's Tuesday! We got hit hard with a load of spam late last night, but it appears to have been relatively unsophisticated. All deletion of spam that is spam is always appreciated etc.