Import not working?

DiscussãoBug Collectors

Entre no LibraryThing para poder publicar.

Import not working?

1jjmcgaffey
Jun 29, 2023, 1:13 am

I imported a CSV file with 48 books in it, last night. No ISBNs, a couple duplicates with books I already have, but mostly importing the text in the file. Nothing has shown up in my library in nearly 24 hours. Does the queue need kicking again?

Related - is there any way I can see progress (or lack of same) on those books? I can go look at the queue, but that's a queue of the ISBNs which I deliberately remove so that what gets entered is my chosen data (which is to say, it remains 0 in the queue at all times).

I'm perfectly willing to re-import - but not if the previous file will then get kicked loose and give me duplicates. Suggestions?

2CtrSacredSciences
Jun 29, 2023, 2:33 am

I've been noticing this the last couple of weeks as well. It's been a while since I did much importing. I notice they hang at night. Some times they add 3-4 days later. Sometimes never. I've wondered if some sources close down at night? Alway have no ISBN, and typically use Overcat / LC / Prospector. Wonder if LC closes at night.

3jjmcgaffey
Jun 30, 2023, 1:49 am

Still nothing 6/29, 48 hours later.

4jjmcgaffey
Jun 30, 2023, 1:50 am

Wonder if some server in LT's array shuts down overnight...

5kristilabrie
Jun 30, 2023, 1:44 pm

>1 jjmcgaffey: I'm taking a look and asking for ccatalfo's help! I don't see any stuck imports, so there might be something else at play. Your import is showing up as still processing, which is odd.

6kristilabrie
Jun 30, 2023, 1:47 pm

Okay, I guess it is getting stuck in the processing stage, we'll need to dig deeper. Hold, please. (Resubmitting the import will likely not help.)

7jjmcgaffey
Jul 1, 2023, 12:30 am

OK, thanks!

I don't _think_ there's any weird characters in there, but there may be.

8CtrSacredSciences
Jul 1, 2023, 2:19 am

>6 kristilabrie: Having this same issue intermittently. Had a batch of 10 go through at 7:27 pm PDT on 6/30/23 but my second batch about 30 mins later is hung. Like the OP, they are all no-ISBN works, so nothing shows in the queue. Thanks.

9kristilabrie
Jul 3, 2023, 9:02 am

>8 CtrSacredSciences: Thanks for the info. For the import that was successful, were they also non-ISBN works?

10CtrSacredSciences
Editado: Jul 3, 2023, 3:53 pm

>9 kristilabrie: Yes all non-ISBN. I have three batches hung now. 10 items, 9 items and 35 items.

11CtrSacredSciences
Jul 3, 2023, 3:53 pm

>9 kristilabrie: Should I just upload them again? Thanks.

12kristilabrie
Jul 6, 2023, 8:44 am

>11 CtrSacredSciences: I don't think that will help at all, ccatalfo is going to try to do some digging in the next week or so to nail this down! We'll post here with any updates. I appreciate your patience in the meantime.

13lasiepedimore
Jul 11, 2023, 9:05 am

Hi, I'm also having problems with importing from Goodreads: I tried the first time on June 27, but nothing happened. So I tried again on July 9, but the library is still empty.

14kristilabrie
Jul 11, 2023, 2:45 pm

>13 lasiepedimore: Thanks for the report, we'll take a look.

15jjmcgaffey
Jul 21, 2023, 6:18 pm

Is it still processing? I've got another batch I want to put up. Any progress?

16kristilabrie
Jul 24, 2023, 9:35 am

>15 jjmcgaffey: It's still stuck at processing, and ccatalfo is out today so I can't get an update from him. You can probably try to re-import your file, and/or upload a new one, to see if that goes through. Let us know what you get, if you can, and I'll ping ccatalfo about this for when he's back!

Thanks for your patience.

17jjmcgaffey
Jul 24, 2023, 5:03 pm

OK, I'll try the extended file - the ones I put up before and the newer books.

18jjmcgaffey
Jul 24, 2023, 5:14 pm

Huh! It's importing as we speak - very quick. I don't see the other one but this one is working...and even more interesting, it's the same file, so it's not that something in the file stuck it up. Unless it terminated wrong somehow. WEIRD. Thanks!

19kristilabrie
Jul 25, 2023, 8:05 am

>18 jjmcgaffey: Okay, great! Let's just close this for now, since whatever was broken seems to have been temporary... and hope it doesn't crop up again. :)

20lasiepedimore
Jul 25, 2023, 8:10 am

>14 kristilabrie: I'm trying to reimport my Goodreads library too, but so far nothing has happened: but it's quite big, so I will wait a while to see if it works this time.

21jjmcgaffey
Set 16, 2023, 1:11 pm

It's doing it again - I successfully imported 67 books on the 14th, attempted almost immediately after to import 4 more - and nothing has happened with the four. I gave it a day and tried to reimport, still nothing. Is it stuck in processing again?

Also, if you're busy trying to keep the site safe and functioning, please ignore this until things settle down!

22kristilabrie
Set 18, 2023, 7:51 am

>21 jjmcgaffey: I see your two most recent imports with 4 records stuck at the same stage, asking ccatalfo about it.

23jjmcgaffey
Set 21, 2023, 2:10 pm

Third time's the charm, I tried again and the latest attempt succeeded in importing 4 books.

24jjmcgaffey
Dez 3, 2023, 2:16 am

Looks like this has come back again. Two attempts, on 11/29 and 12/1, and I just tried for a third time on 12/2. Nothing coming through yet. Same file each time, 49 books without ISBNs.

25waltzmn
Dez 3, 2023, 3:06 am

>24 jjmcgaffey:

Trying the same file each time means that you aren't varying any parameters. How can you tell if it's LT as opposed to the file? What happens if you try a different file, or break the file into two parts and import the two parts separately?

I realize that if there is a bug, it's not your fault. But if you really want to demonstrate the bug, you need to test multiple scenarios.

26jjmcgaffey
Editado: Dez 3, 2023, 2:43 pm

Scroll up - I've tried multiple ways over the months this has been intermittently showing up. The only thing that has consistently worked (that is, finally got the file to import) is re-submitting a few days apart. So I tried that before reporting the (return of the) bug. I also haven't reported a couple times when resubmitting _did_ work.

27kristilabrie
Dez 4, 2023, 8:24 am

>24 jjmcgaffey: Looks like non-ISBN imports got stuck again, even though it looked like they were still processing on our end. (We need to find and fix that.)

ccatalfo kickstarted your first attempt and it looks like it went through, if you want to have a look?

28jjmcgaffey
Dez 4, 2023, 3:03 pm

Got all 49, thanks!

29kristilabrie
Dez 5, 2023, 7:28 am

Great, keeping this open for now since we still need to pinpoint the issue, but at least we could finish your import for you!

30simonjbale
Dez 5, 2023, 11:19 am

i'm getting it. I have a rather large 1809 library. Got to 940 and just stuck. No messages, no nothing... Any ideas?

31MarthaJeanne
Dez 5, 2023, 12:15 pm

There are lots of things that can go wrong with an import, but you haben't told us much.

How many books?

ISBN look-up or not?

If it's a look-up, what source?

Are you sure all the columns are correct?

32waltzmn
Dez 5, 2023, 3:15 pm

>30 simonjbale: Got to 940 and just stuck. No messages, no nothing... Any ideas?

The obvious thing to do is to break the import up into smaller pieces. That many books, the chance of bad data is quite high. If 940 was the problem, start a new import at 941 (or 950, or 1000, or something), and see how it does. Then look at 941 to see if there is something wrong with it, or enter it yourself.

I would think this a good idea anyway, for those doing a lot of imports. We know that the LT import mechanism is a little flaky, but it is also dependent on good data. Don't try to import everything you own all at once! -- not when one error can ruin the whole process. Import 50 or 100 or 200 books at a time and see if it works, then enter the next batch.

33AnnieMod
Dez 5, 2023, 3:22 pm

>32 waltzmn: I'd also look at the last book that DID import before it stopped - it may be corrupted if that is the line breaking something and it may have managed to import incompletely.

34waltzmn
Dez 5, 2023, 3:41 pm

>33 AnnieMod: I'd also look at the last book that DID import before it stopped - it may be corrupted if that is the line breaking something and it may have managed to import incompletely.

Absolutely agree. The point is just, Do the import in small pieces, so you know where the problems are, and then look at the problem parts.

35jjmcgaffey
Dez 21, 2023, 11:59 pm

And again - imported 77 books last night, nothing as of tonight. No ISBNs. Is that what's getting stuck?

36LucindaLibri
Editado: Dez 31, 2023, 2:29 pm

Same problem here. Was only trying 8 books. All without ISBNs. At this point I should have just manually entered them using copy and paste from my carefully constructed file. Uggggh

I did notice that when it showed me my 8 books, it only showed the first 3 columns. Has the "'PUBLICATION INFO'" column changed its name? I'm just using the sample file but maybe that is out of date.

UPDATE: Whoops. Looks like the second time it may have worked. (I omitted the duplicates in the second file.)
So if you don't use ISBNs, the info about where you are in the queue is always "you don't have anything in the queue?"

37jjmcgaffey
Jan 9, 10:33 pm

Yep, if there's no ISBN there's no ISBN queue. And it looks like it's the no-ISBN imports that get blocked somewhere along the line. I have gotten them to show up twice now by re-importing after a couple days, so that may be what worked (whether you changed the contents of the file or not).

I don't think I've ever had it show me my books - I did a 4-book import a while ago and it got stuck. Where/when did it show you the first three columns?

38jjmcgaffey
Jan 18, 2:22 pm

And stuck again - I sent the file several times over the last week or so and nothing. Any ideas on where it's getting stuck? I recall that for a while someone had to go kick the ISBN queue on a regular basis, is there something similar happening with the non-ISBN queue now?

39kristilabrie
Jan 19, 8:00 am

Sorry for the import trouble! I'll try taking a look this morning and see what I (or ccatalfo) can do. I'll report back.

40kristilabrie
Jan 19, 11:21 am

>38 jjmcgaffey: Thanks for the wait. Looks to be the same issue (the imports needed a kick)... we'll need to dive further into this when we have time but ccatalfo is aware of the issue.

Your latest import should be kicked and processing, now! Let me know if there's anything else I can do in the meantime.

41jjmcgaffey
Jan 20, 1:40 am

Got...one of them, oddly not the latest. But that's fine, it's the same contents. Thanks!

42junebug75
Jan 23, 8:14 pm

I am having the same issue. Uploaded my first file (with just two books) as a test and 24 hours later I still have no books. I am hoping to upload my nearly 4,000 book collection (none of which has any ISBNs). I am using the csv template provided by LibraryThing.

43kristilabrie
Jan 24, 8:07 am

>42 junebug75: We really need to dig into the non-ISBN imports soon to figure this out, but we're kicking your 2-book upload in the meantime. Please check back on those soon! If you need help kicking the final 4K upload, when you get to it, just let me know. (You can also DM me kristilabrie, from my profile, or email me at kristiATlibrarythingDOTcom.)

44vickila49
Jan 31, 12:39 am

I uploaded to worksheets last week the I got off an old thumb drive. I had to load the drive on a newer Mac in numbers, convert to excel, move excel to my Mac book air, change it to CSV and then it seemed to upload.

I got an email that one worksheet had been received the same day as the upload. But only the names of the books have appeared on my book list.

No word yet on the second larger worksheet ( 900 plus books).

I think I did the columns correctly. I joined an electronics club so I could have some IT get me to this point.

No ISBN numbers..

What next?

45kristilabrie
Fev 2, 9:37 am

>44 vickila49: Thanks for your patience with my response! I took a quick look at your import files and it looks like your headers are not matching LibraryThing's sample .csv file exactly. Please be sure to read over our Help page for how to format your .csv file to match our sample import file exactly (https://wiki.librarything.com/index.php/HelpThing:Import): you must follow the exact same name, order, and spelling of the column headers in the top row—no more and no less—in order to import everything properly. I hope this helps!

46r.orrison
Fev 2, 4:28 pm

>45 kristilabrie: I took a quick look at your import files and it looks like your headers are not matching LibraryThing's sample .csv file exactly

It would be a quick check to make sure the column headers were correct before submitting, and reject the file immediately with an appropriate message. (Or possibly warn, with an option to search the file for ISBNs.)

https://www.librarything.com/topic/290292

47vickila49
Fev 2, 7:50 pm

Ok. I will try again. I did have a second set of eyes look at this. An retired IT person, but I guess we both missed the miss match. Can you give me a hint? About what’s wrong. I have some dyslexia and am old.

Thank you!!

48jjmcgaffey
Fev 3, 12:24 am

>47 vickila49: On the Universal Import page, at the bottom of the right sidebar, there's a sample CSV file (LibraryThingSample.csv). That one is definitely correct, so it's the best template to use for yours - that's what I did. Capitalization, single (straight, not curly) quotes, etc. Much easier to get a copy than try to make it from scratch.

49vickila49
Fev 4, 11:47 am

I did us the sample. All caps. I could not get the single quotation marks to stick. Do they need to be there. Example: ‘PAGE COUNT’

50jjmcgaffey
Fev 5, 2:25 am

Yes - but 'PAGE COUNT' - straight quotes, not curly/slanted. It makes a lot of difference to a computer. And the easiest way to do it is to make a copy of the sample and insert your data into it.

The way I do it (I'm importing a list of my ebooks from calibre) - generate a catalog with the proper data in the proper order. Open the spreadsheet that results. Copy the columns (all one chunk, since they're already in the right order). Paste them into another spreadsheet that started as a copy of the LibraryThingSample - I never touch the headers at all, only the cells below them. Save that spreadsheet, upload it to LT.

I deliberately leave the ISBN field blank (delete any ISBNs that appear in the file-to-be-uploaded) because a) a lot of my books don't have them and b) I want my data to match what I have in calibre, not whatever LT digs up for that ISBN. Your choice on that.

51vickila49
Fev 6, 2:36 pm

It worked! Thank you!
The single quotation mark thing threw me off because when I relabeled the columns the first quotation mark always disappeared. I submitted the spread sheets with the correct heading ,but no quotation marks and for some reason one spread sheet up loaded with just titles. Today,
I retyped all the headings with the straight quotation marks today and even though the first quotation mark doesn’t show on my spread sheet, it must be there because it uploaded.

52newcrossbooks
Fev 6, 4:44 pm

>51 vickila49: The single quotation mark ' is a special character for Excel when it appears as the first character in a cell. It tells Excel to treat the rest of the string as text and so won't show on your spreadsheet, though it will still be visible in the data entry box for the cell.

It's useful if you want to be able to display in full a data entry that has leading zeros on your spreadsheet. Telephone numbers, zip codes, catalogue codes, etc starting with 0 will show in full if you start the entry with a ' as this forces the number to display as text.

53jjmcgaffey
Fev 6, 9:52 pm

Ah, I'm using Calc - which does have the single quote thing, but apparently doesn't do it if there is another single quote at the end of the cell. Or something. It certainly shows them in my headers (and I've used a single quote to make some 10-digit ISBNs behave, among other things).

54vickila49
Fev 11, 3:36 pm

I uploaded two spreadsheets in csv with the 11 columns from the universal upload sample spread sheet. I put the single quotations on each column heading. 'TITLE' ETC. I got confirmation that the uploads when through, but on checking the site, I found only the 'TITLE', 'DATE', 'PUBLICATION INFO'. AND 'TAGS' COLUMNS INFO appeared. 'AUTHOR', 'RATING' AND 'REVIEW' 'DATE READ', 'PAGE COUNT' and 'CALL NUMBER' were left blank

I checked load without ISBN.

Why doesn't any of the data in the other columns load?

55jjmcgaffey
Fev 12, 2:29 pm

Weird! I've never had a partial success. I don't upload rating so I don't know what format it wants; Date Read should be the same as Date (yyyy-mm-ddThh:mm:ss(time zone), truncated as you please from the right). But Page Count should be just an integer and Call Number, Review, and certainly Author should be free text. The columns were in the same order as the sample sheet?

No idea. When one of the LT people responds here again, you might offer to give them your spreadsheet so they can look and see where the program is falling down. Or DM kristilabrie and ask if someone can take a look at it.

56vickila49
Fev 12, 8:15 pm

Thank you for responding.
I would be have to send the spread sheet to anyone who would like to look. I don't see what I am doing wrong unless as you say each column has a special format.

The dates entered with just the year.

"DM" kristilabrie means?

57bnielsen
Fev 13, 8:01 am

DM = Direct Message, i.e. use the LT message system or send her an email.

58vickila49
Fev 13, 12:41 pm

Kristilabrie ? Doesn’t seem to be an Email address. How do I find that?
How do I direct message kirstilabrie?

59AnnieMod
Fev 13, 12:58 pm

>58 vickila49: She had posted above but here is her profile: https://www.librarything.com/profile/kristilabrie

There is a "Message" button in there in the right menu.

60norabelle414
Fev 13, 2:49 pm

>58 vickila49: You can also click "Contact" at the very bottom of any page to find all the staff email addresses.

61ECCClibrary
Fev 28, 1:53 pm

I have imported a file (Uploaded csv20240216) with 58 items on Feb. 17th, I waited and checked on Feb. 26th - nothing have been added to my collection. I imported it again, still the same. I thought may be my file is too big (?)

So I separated the first file into two files (Uploaded csv20240216A and Uploaded csv20240216b) and submitted them on Feb. 26th, still nothing happened.

Yesterday, I separated the file (Uploaded csv20240216A) into 2. I tried to test and just submitted the newly separated file (Uploaded csv20240216Aa), still nothing happened this morning.

May I ask what's the problem? My file? or the system? Thanks.

62waltzmn
Fev 28, 2:05 pm

>61 ECCClibrary: May I ask what's the problem? My file? or the system? Thanks.

Someone will probably have a better answer soon, but the first thing to do is to look at your file in extreme detail. It must match the example file exactly.

63ECCClibrary
Fev 28, 2:21 pm

Yes, I think it was correct because I copy and paste data to each column of the LibrarySample csv file. I have uploaded files many times since October. This kind of problem only happened a couple of times.

I have submitted a file with 10 items on Feb. 24th. It was imported successfully.

Our library is working on database migration. We had submitted files, each with over 100 items before. They were okay.

64SandraArdnas
Fev 28, 7:00 pm

>63 ECCClibrary: Contact Kristy from the contacts at the bottom of each page. She'll be able to see where and why it's stuck

65kristilabrie
Editado: Fev 29, 8:37 am

>61 ECCClibrary: Kristi here! I'll take a look, hold please.

Update: Okay, I think this is more non-ISBN import issues that we still need to dig deep into. They've been having trouble stalling. I'm asking ccatalfo to check on your original import from February 17 and kick start it if needed.

66kristilabrie
Fev 29, 8:49 am

Okay, your import from Feb 17 should be running ECCClibrary! I assume your other imports won't end up running, but you may want to keep an eye on your catalog for the next few days in case they come in and you want to delete any duplicate records. I hope this helps.

67ECCClibrary
Fev 29, 1:17 pm

68jjmcgaffey
Fev 29, 2:46 pm

Yeah - if the file is right (and if it's been working before, it's right - there haven't been any recent changes) then the problem is the system. I recall when the LT people used to have to kick the ISBN queue on a(n ir)regular basis, and they fixed that, so hopefully they can figure out where the non-ISBN queue is getting choked up.

I have successfully imported 1400+ books (not recently), so really the size of the file doesn't have a lot of effect (takes a bit longer, a few hours instead of minutes...). I regularly import 20-80 books at a time.

69scraps
Mar 8, 5:19 pm

>47 vickila49: Hi, I found an easy way was to copy the first (titles/headers) line into a new spread sheet. Then I copied the relevant information from my original excel file into the appropriate columns. final step was to save with the original name_import.csv

70jjmcgaffey
Mar 17, 4:09 pm

FYI - I imported a number of books (56 or so) on March 7th; no reaction, they weren't added, nothing. Today (March 17) I imported 93 books (the original 50+ and some new ones); they were added almost instantly (5 minutes maybe). Identical file format, no ISBNs, identical procedure. Has there been a fix made, or did I just get lucky?

71kristilabrie
Mar 18, 9:11 am

>70 jjmcgaffey: I think the latter, alas (or thankfully?).. no major changes have been made to the import yet, but it is on our list of things to look at, hopefully during our next bug cleanup in the near future. Thanks for your patience; I'm glad the 2nd round worked well for you!

72vickila49
Abr 16, 10:23 am

I uploaded about 980 books about 10 days ago. The upload page says I was successful. I see nothing yet in my books. I looked at the Queue box on the upload page. It says I have nothing in the queue. None of my books had ISBN numbers. Do I keep waiting?

73kristilabrie
Abr 16, 2:07 pm

>72 vickila49: I see one from April 10, I'll see if ccatalfo can give it a kick for you. Try checking back in a bit?

74vickila49
Abr 16, 6:21 pm

Wow. It's all there. A dream come true to get all this catalogued. Great service!
Many thanks!

75kristilabrie
Abr 17, 7:31 am

>74 vickila49: Great! If you run into this again just let us know.

76vickila49
Abr 17, 8:45 am

Me again.

I want to show my library to a friend. I looked at the instructions. Went to Safari: typed in http://www.librarything.com/profile/vicila49. My library appeared but it looks like I could alter by accessing it this way.

Is there a way to allow a friend to look but not change?

77AnnieMod
Editado: Abr 17, 8:47 am

>76 vickila49: You can change things because it is your library. Other users cannot and see it in read only mode.

Log out or use a different browser or an anononymous window to look at your library to see what other members see.

78vickila49
Abr 17, 7:10 pm

Thank you. I get it now.