UPGRADE YOUR SKILLS: Learn advanced Swift and SwiftUI on Hacking with Swift+! >>

Import large dataset to Core Data

Forums > SwiftUI

I have a large dataset as 300.000 entities consisting of two string attributes: bookName, bookAuthor and bookCategory. The app would be first display 100 books on the first time user open the app.

So I plan to import 100 in main context, and the rest would be imported in the background context. My questions are:

  1. Is there any issues to fetch 100 books at the first time to display on the screen? How would we do it?

  2. There are about a hundred of bookCategory. Is it efficient to fetch books based on bookCategory to display if i use dynamically filter based on category? Or I need to set relationship like Book and CategoryBook?

  3. How would we optimize this import process?

3      

Assuming that your Core Data entity is a Book containing the attributes bookName, bookTitle & bookCategory.


@EnvironmentObject var firstTimeDisplayed: Bool

let bookList: FetchRequest<Book>

  init() {

  let bookRequest: NSFetchRequest<Book> = Book.fetchRequest()

  bookRequest.sortDescriptors = [NSSortDescriptor(keyPath: \Book.title, ascending: true)]

  if firstTimeDisplayed {
      bookRequest.fetchLimit = 100
  }

  bookList = FetchRequest(fetchRequest: bookRequest)
  }

  ... 

  // after determining that 100 books have been displayed for the first time
  firstTimeDisplayed = false

I have split out the request so that fetchLimit is visible.

Also the firstTimeDisplayed parameter needs to keep its value even when the app is closed and restarted, so EnvironmentObject is a possible candidate.

The sorting is shown using a single keypath, and can be extended to use multiple keypaths - multiple sort keypaths

Depending on how quickly the initial 100 books are imported, you may need to keep a track of the number of book imported and then update the view from a holding view ('Setting up', 'Importing') to the display of the first 100 books.

I have not tried this out, so may need some tweaking. I hope that this helps you and gives you some ideas on how to proceed.

4      

@Greenamberred thank you for your suggestion.

For the app, users can not write/add books into core data. So what do you recommend for the model data. Should we build the data model with:

  • 2 entites: Book Entity and BookCategory Entity
  • or just one Book entity with category attribute that can be filter by using predicate

3      

I assumed that the user was not writing in to the Core Data, and that your app was importing it from a JSON or CSV or similar file.

The fetchRequest, etc is for the View context to display items in the Core Data.

I think that it is only natural to have one entity - 'Book', unless you plan to do something else with bookCategory that has nothing to do with either the bookAuthor or bookName.

4      

Thats right, user was not writing in to the Core Data, and that t the app was importing it from a JSON file.

If I have like 300,000 record of book, stored in JSON file, what do you think the best way to preload data to the app?

I plan to import 100 books in main context, and the rest would be done in the background context

3      

Not sure why you would want to separate preloading and loading in the background. Why not just load?

Once there are 100 books loaded into Core Data, you would display the 100 books and carry on populating the remaining books into the Core Data.

You might need a temporary display of 'setting up' whilst the first 100 books are loaded in to Core Data, depending on the device's speed, but probably not. It will likely be quick enough for 100 books.

3      

Hacking with Swift is sponsored by RevenueCat

SPONSORED Take the pain out of configuring and testing your paywalls. RevenueCat's Paywalls allow you to remotely configure your entire paywall view without any code changes or app updates.

Learn more here

Sponsor Hacking with Swift and reach the world's largest Swift community!

Archived topic

This topic has been closed due to inactivity, so you can't reply. Please create a new topic if you need to.

All interactions here are governed by our code of conduct.

 
Unknown user

You are not logged in

Log in or create account
 

Link copied to your pasteboard.