热门标签 | HotTags
当前位置:  开发笔记 > 编程语言 > 正文

在Android中预先填充数据库的最快,最有效的方法-Fastestandmostefficientwaytopre-populatedatabaseinAndroid

Ifyouwanttopre-populateadatabase(SQLite)inAndroid,thisisnotthateasyasonemightthink

If you want to pre-populate a database (SQLite) in Android, this is not that easy as one might think.

如果你想在Android中预先填充数据库(SQLite),这并不像人们想象的那么容易。

So I found this tutorial which is often referenced here on Stack Overflow as well.

所以我发现这个教程也经常在Stack Overflow上引用。

But I don't really like that way of pre-populating the database since you take the control from the database handler and create the files yourself. I would prefer to not touch the file system and let the database handler do everything on its own.

但是我不喜欢这种预先填充数据库的方式,因为您从数据库处理程序中获取控件并自己创建文件。我宁愿不触摸文件系统,让数据库处理程序自己完成所有事情。

So what I thought one could do is create the database in the database handler's onCreate() as usual but then load a file (.sql) from /assets which contains the statements to fill in the values:

所以我认为可以做的就是像往常一样在数据库处理程序的onCreate()中创建数据库,然后从/ assets加载一个文件(.sql),其中包含要填充值的语句:

INSERT INTO testTable (name, pet) VALUES ('Mike', 'Tiger');
INSERT INTO testTable (name, pet) VALUES ('Tom', 'Cat');
...

But calling execSQL() in the handler's onCreate() doesn't really work. It seems that the /assets file must not have more than 1MB and the execSQL() only executes the first statement (Mike - Tiger).

但是在处理程序的onCreate()中调用execSQL()并不真正起作用。似乎/ assets文件不能超过1MB而execSQL()只执行第一个语句(Mike-Tiger)。

What would you do do pre-populate the database?

你会做什么做预先填充数据库?

7 个解决方案

#1


5  

I suggest the following:

我建议如下:

  1. Wrap all of your INSERT logic into a transaction (BEGIN... COMMIT, or via the beginTransaction()... endTransaction() APIs)
  2. 将所有INSERT逻辑包装到事务中(BEGIN ... COMMIT,或通过beginTransaction()... endTransaction()API)
  3. As already suggested, utilize the bind APIs and recycle objects.
  4. 如前所述,使用绑定API和回收对象。
  5. Don't create any indexes until after this bulk insert is complete.
  6. 在批量插入完成之前,不要创建任何索引。

Additionally take a look at Faster bulk inserts in sqlite3?

另外看一下sqlite3中的Faster批量插入?

#2


4  

Your question states, that you want the fastest way - but you don't like the way it's done in the article - you don't want to manually replace the DB file (even though, it may be actually faster than filling empty DB with queries).

你的问题表明,你想要最快的方式 - 但你不喜欢文章中的方式 - 你不想手动替换数据库文件(尽管它实际上可能比填充空数据库更快)查询)。

I had exaclty the same thoughts - and I figured out, that populating via SQL statements and prepopulating can both be the best solution - but it depends on the way you will use the DB.

我有同样的想法 - 我想通过SQL语句填充和预填充都可以是最好的解决方案 - 但这取决于你使用数据库的方式。

In my application I need to have about 2600 rows (with 4 columns) in DB at the very first run - it's the data for autocompletion and few other things. It will be modified quite rarely (users can add custom records, but most of the time - they don't need to) and is quite big. Populating it from SQL statements takes not only significantly more time, but more space in the APK (assuming I would store data inside it, alternatively I could download it from the internet).

在我的应用程序中,我需要在第一次运行时在DB中有大约2600行(有4列) - 它是自动完成的数据和其他一些东西。它将很少被修改(用户可以添加自定义记录,但大部分时间 - 他们不需要)并且非常大。从SQL语句填充它不仅需要更多的时间,而且还需要更多的空间(假设我将数据存储在其中,或者我可以从互联网上下载)。

This is the very simple case (the "Big" insert can take place only once and only at first startup) and I decided to go with copying prepopulated DB file. Sure, it may not be the nicest way - but it's faster. I want my users to be able to use the app as quickly as it's possible and treat speed as a priority - and they really like it. On the contrary, I doubt they would be glad when app would slow down because I thought that slower and nicer solution is actually better.

这是一个非常简单的情况(“大”插入只能在第一次启动时发生一次)我决定使用复制预先填充的DB文件。当然,它可能不是最好的方式 - 但它更快。我希望我的用户能够尽可能快地使用应用程序并将速度视为优先事项 - 他们非常喜欢它。相反,我怀疑当应用程序减速时他们会很高兴,因为我认为更慢更好的解决方案实际上更好。

If instead of 2600 my table would have initially ~50 rows, I would go with SQL statements, since speed and size difference wouldn't be so big.

如果不是2600我的表最初会有~50行,我会使用SQL语句,因为速度和大小差异不会那么大。

You have to decide which solution fits your case better. If you foresee any problems that may arise from using "prepopulated db" option - don't use it. If you are not sure about these problems - ask, providing more details on how you will use (and eventually, upgrade) contents of the DB. If you aren't quite sure which solution will be faster - benchmark it. And don't be afraid of that copying file method - it can work really well, if used wisely.

您必须更好地决定哪种解决方案适合您的情况。如果您预见到使用“预填充db”选项可能引起的任何问题 - 请勿使用它。如果您不确定这些问题 - 请询问,提供有关如何使用(并最终升级)数据库内容的更多详细信息。如果你不确定哪种解决方案会更快 - 那么就要对它进行基准测试。并且不要害怕复制文件方法 - 如果明智地使用它可以很好地工作。

#3


2  

You can have your cake and eat it too. Here is a solution that can both respect the use of your db adapter and also use a simple (and much faster) copy process for a pre-populated database.

你可以吃蛋糕,也可以吃。这是一个解决方案,它既可以尊重数据库适配器的使用,也可以对预先填充的数据库使用简单(且速度更快)的复制过程。

I'm using a db adapter based on one of Google's examples. It includes an internal class dbHelper() that extends Android's SQLiteOpenHelper() class. The trick is to override it's onCreate() method. This method is only called when the helper can't find the DB you are referencing and it has to create the DB for you. This should only happen the first time it is called on any given device installation, which is the only time you want to copy the DB. So override it like this -

我正在使用基于Google的一个示例的数据库适配器。它包含一个内部类dbHelper(),它扩展了Android的SQLiteOpenHelper()类。诀窍是覆盖它的onCreate()方法。只有在帮助程序找不到您正在引用的数据库且必须为您创建数据库时才会调用此方法。这应该仅在第一次在任何给定的设备安装上调用时发生,这是您唯一要复制数据库的时间。所以像这样覆盖它 -

    @Override
    public void onCreate(SQLiteDatabase db) {
        mNeedToCopyDb = true;
    }

Of course make sure you have first declared and initialized this flag in the DbHelper -

当然要确保你在DbHelper中首先声明并初始化了这个标志 -

        private Boolean mNeedToCopyDb = false;

Now, in your dbAdapter's open() method you can test to see if you need to copy the DB. If you do then close the helper, copy the DB and then finally open a new helper (see below code). All future attempts to open the db using the db adapter will find your (copied) DB and therefor the onCreate() method of the internal DbHelper class will not be called and the flag mNeedToCopyDb will remain false.

现在,在dbAdapter的open()方法中,您可以测试是否需要复制数据库。如果你这样做,然后关闭帮助程序,复制数据库,然后最后打开一个新的帮助程序(见下面的代码)。将来使用db适配器打开数据库的所有尝试都将找到您的(复制的)DB,因此不会调用内部DbHelper类的onCreate()方法,并且标志mNeedToCopyDb将保持为false。

    /**
 * Open the database using the adapter. If it cannot be opened, try to
 * create a new instance of the database. If it cannot be created,
 * throw an exception to signal the failure.
 * 
 * @return this (self reference, allowing this to be chained in an
 *         initialization call)
 * @throws SQLException if the database could neither be opened nor created
 */
public MyDbAdapter open() throws SQLException {
    mDbHelper = new DatabaseHelper(mCtx);
    mDb = mDbHelper.getReadableDatabase();

    if (mDbHelper.mNeedToCopyDb == true){
        mDbHelper.close();
        try {
            copyDatabase();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            mDbHelper = new DatabaseHelper(mCtx);
            mDb = mDbHelper.getReadableDatabase();
        }
    }
    return this;
}

Just place some code to do your database copy inside of your db adapter in a method named copyDatabase() as used above. You can use the value of mDb that was updated by the first instance of DbHelper (when it created the stub DB) to get the path to use for your output stream when you do the copy. Construct your input stream like this

只需放置一些代码,就可以在上面使用的名为copyDatabase()的方法中在db适配器内部进行数据库复制。您可以使用由DbHelper的第一个实例(创建存根数据库时)更新的mDb值,以便在执行复制时获取用于输出流的路径。像这样构造输入流

dbInputStream = mCtx.getResources().openRawResource(R.raw.mydatabase);

[note: If your DB file is too large to copy in one gulp then just break it up into a few pieces.]

[注意:如果你的数据库文件太大而无法一次性复制,那么就把它分成几块。]

This works very fast and puts all of the db access code (including the copying of the DB if needed) into your db adapter.

这非常快,并将所有数据库访问代码(包括数据库的复制,如果需要)放入数据库适配器。

#4


2  

I wrote a DbUtils class similar to the previous answer. It is part of the ORM tool greenDAO and is available on github. The difference is that it will try to find statement boundaries using a simple regular expression, not just line endings. If you have to rely on a SQL file, I doubt that there's a faster way.

我写了一个类似于上一个答案的DbUtils类。它是ORM工具greenDAO的一部分,可以在github上找到。不同之处在于它将尝试使用简单的正则表达式来查找语句边界,而不仅仅是行结尾。如果你必须依赖SQL文件,我怀疑有更快的方法。

But, if you can supply the data in another format, it should be significantly faster than using a SQL script. The trick is to use a compiled statement. For each data row, you bind the parsed values to the statement and execute the statement. And, of course, you need to do this inside a transaction. I would recommend a simple delimiter separated file format (for example CSV) because it can be parsed faster than XML or JSON.

但是,如果您可以以其他格式提供数据,那么它应该比使用SQL脚本快得多。诀窍是使用编译语句。对于每个数据行,将解析后的值绑定到语句并执行该语句。当然,您需要在事务中执行此操作。我建议使用简单的分隔符分隔文件格式(例如CSV),因为它可以比XML或JSON更快地解析。

We did some performance tests for greenDAO. For our test data, we had insert rates of about 5000 rows per second. And for some reason, the rate dropped to half with Android 4.0.

我们为greenDAO做了一些性能测试。对于我们的测试数据,我们的插入速率约为每秒5000行。出于某种原因,Android 4.0的速度降至一半。

#5


0  

ye, the assets maybe has size limit, so if bigger than the limit, you can cut to more files.

是的,资产可能有大小限制,所以如果大于限制,你可以削减更多的文件。

and exesql support more sql sentence, here give you a example:

和exesql支持更多的sql语句,这里举个例子:

    BufferedReader br = null;
    try {
        br = new BufferedReader(new InputStreamReader(asManager.open(INIT_FILE)), 1024 * 4);
        String line = null;
        db.beginTransaction();
        while ((line = br.readLine()) != null) {
            db.execSQL(line);
        }
        db.setTransactionSuccessful();
    } catch (IOException e) {
        FLog.e(LOG_TAG, "read database init file error");
    } finally {
        db.endTransaction();
        if (br != null) {
            try {
                br.close();
            } catch (IOException e) {
                FLog.e(LOG_TAG, "buffer reader close error");
            }
        }
    }

above example require the INIT_FILE need every line is a sql sentence.

上面的例子要求INIT_FILE需要每一行都是一个sql语句。

Also, if your sql sentences file is big, you can create the database out site of android(sqlite support for windows, linux, so you can create the database in your os, and copy the database file to your assets folder, if big, you can zip it)

另外,如果您的sql语句文件很大,您可以创建android的数据库站点(sqlite支持windows,linux,因此您可以在您的操作系统中创建数据库,并将数据库文件复制到您的资源文件夹,如果大,你可以压缩它)

when your application run, you can get the database file from assets, directed to save to your application's database folder (if you zip it, you can unzip to the application's database folder)

当您的应用程序运行时,您可以从资产中获取数据库文件,并将其保存到应用程序的数据库文件夹中(如果您将其压缩,则可以解压缩到应用程序的数据库文件夹)

hope can help you -):

希望可以帮到你 - ):

#6


0  

I used this method. First create your sqlite database there are a few programs you can use I like SqliteBrowser. Then copy your database file into your assets folder. Then you can use this code in the constructor of SQLiteOpenHelper.

我用过这种方法。首先创建你的sqlite数据库你可以使用一些程序我喜欢SqliteBrowser。然后将数据库文件复制到assets文件夹中。然后,您可以在SQLiteOpenHelper的构造函数中使用此代码。

final String outFileName = DB_PATH + NAME;

        if(! new File(outFileName).exists()){
            this.getWritableDatabase().close();
            //Open your local db as the input stream
            final InputStream myInput = ctx.getAssets().open(NAME, Context.MODE_PRIVATE);

            //Open the empty db as the output stream
            final OutputStream myOutput = new FileOutputStream(outFileName);
            //final FileOutputStream myOutput = context.openFileOutput(outFileName, Context.MODE_PRIVATE);

            //transfer bytes from the inputfile to the outputfile
            final byte[] buffer = new byte[1024];
            int length;
            while ((length = myInput.read(buffer))>0){
                myOutput.write(buffer, 0, length);
            }

            //Close the streams
            myOutput.flush();
            ((FileOutputStream) myOutput).getFD().sync();
            myOutput.close();
            myInput.close();
        }
        } catch (final Exception e) {
            // TODO: handle exception
        }

DB_PATH is something like /data/data/com.mypackage.myapp/databases/

DB_PATH类似于/data/data/com.mypackage.myapp/databases/

NAME is whatever database name you choose "mydatabase.db"

NAME是您选择的任何数据库名称“mydatabase.db”

I know there are many improvements on this code but it worked so well and is VERY FAST. So I left it alone. Like this might be even better in the onCreate() method. Also checking if the file exists every time is probably not the best. Anyway like I said it works, it's fast and reliable.

我知道这个代码有很多改进但它运行得很好并且非常快。所以我一个人待着。像这样在onCreate()方法中可能会更好。另外,每次检查文件是否存在可能不是最好的。无论如何,就像我说它有效,它快速可靠。

#7


0  

If the data is not private then simply host it on your website then download it on first run. That way you can keep it up to date. So long as you remember to take app version into account when you upload it to your webserver.

如果数据不是私有的,那么只需将其托管在您的网站上,然后在首次运行时下载。这样你可以保持最新。只要您记得在将应用版本上传到网络服务器时将其考虑在内。


推荐阅读
  • MyBatis多表查询与动态SQL使用
    本文介绍了MyBatis多表查询与动态SQL的使用方法,包括一对一查询和一对多查询。同时还介绍了动态SQL的使用,包括if标签、trim标签、where标签、set标签和foreach标签的用法。文章还提供了相关的配置信息和示例代码。 ... [详细]
  • ALTERTABLE通过更改、添加、除去列和约束,或者通过启用或禁用约束和触发器来更改表的定义。语法ALTERTABLEtable{[ALTERCOLUMNcolu ... [详细]
  • 本文介绍了iOS数据库Sqlite的SQL语句分类和常见约束关键字。SQL语句分为DDL、DML和DQL三种类型,其中DDL语句用于定义、删除和修改数据表,关键字包括create、drop和alter。常见约束关键字包括if not exists、if exists、primary key、autoincrement、not null和default。此外,还介绍了常见的数据库数据类型,包括integer、text和real。 ... [详细]
  • 在Oracle11g以前版本中的的DataGuard物理备用数据库,可以以只读的方式打开数据库,但此时MediaRecovery利用日志进行数据同步的过 ... [详细]
  • HTC EVO 4G+手机存储(SD卡)中各个文件夹功能说明(转载)
      HTCRider/X515E/EVO4G+手机存储(SD卡)中各个文件夹功能说明  HTCRider/X515E/EVO4G+  1、.android_s ... [详细]
  • VScode格式化文档换行或不换行的设置方法
    本文介绍了在VScode中设置格式化文档换行或不换行的方法,包括使用插件和修改settings.json文件的内容。详细步骤为:找到settings.json文件,将其中的代码替换为指定的代码。 ... [详细]
  • 向QTextEdit拖放文件的方法及实现步骤
    本文介绍了在使用QTextEdit时如何实现拖放文件的功能,包括相关的方法和实现步骤。通过重写dragEnterEvent和dropEvent函数,并结合QMimeData和QUrl等类,可以轻松实现向QTextEdit拖放文件的功能。详细的代码实现和说明可以参考本文提供的示例代码。 ... [详细]
  • Linux重启网络命令实例及关机和重启示例教程
    本文介绍了Linux系统中重启网络命令的实例,以及使用不同方式关机和重启系统的示例教程。包括使用图形界面和控制台访问系统的方法,以及使用shutdown命令进行系统关机和重启的句法和用法。 ... [详细]
  • android listview OnItemClickListener失效原因
    最近在做listview时发现OnItemClickListener失效的问题,经过查找发现是因为button的原因。不仅listitem中存在button会影响OnItemClickListener事件的失效,还会导致单击后listview每个item的背景改变,使得item中的所有有关焦点的事件都失效。本文给出了一个范例来说明这种情况,并提供了解决方法。 ... [详细]
  • eclipse学习(第三章:ssh中的Hibernate)——11.Hibernate的缓存(2级缓存,get和load)
    本文介绍了eclipse学习中的第三章内容,主要讲解了ssh中的Hibernate的缓存,包括2级缓存和get方法、load方法的区别。文章还涉及了项目实践和相关知识点的讲解。 ... [详细]
  • 本文介绍了游标的使用方法,并以一个水果供应商数据库为例进行了说明。首先创建了一个名为fruits的表,包含了水果的id、供应商id、名称和价格等字段。然后使用游标查询了水果的名称和价格,并将结果输出。最后对游标进行了关闭操作。通过本文可以了解到游标在数据库操作中的应用。 ... [详细]
  • 本文介绍了在使用Laravel和sqlsrv连接到SQL Server 2016时,如何在插入查询中使用输出子句,并返回所需的值。同时讨论了使用CreatedOn字段返回最近创建的行的解决方法以及使用Eloquent模型创建后,值正确插入数据库但没有返回uniqueidentifier字段的问题。最后给出了一个示例代码。 ... [详细]
  • 涉及的知识点-ViewGroup的测量与布局-View的测量与布局-滑动冲突的处理-VelocityTracker滑动速率跟踪-Scroller实现弹性滑动-屏幕宽高的获取等实现步 ... [详细]
  • Apple iPad:过渡设备还是平板电脑?
    I’vebeenagonizingoverwhethertopostaniPadarticle.Applecertainlydon’tneedmorepublicityandthe ... [详细]
  • 篇首语:本文由编程笔记#小编为大家整理,主要介绍了Mongodb副本集+分片集群搭建相关的知识,希望对你有一定的参考价值。环境需求: ... [详细]
author-avatar
周家华099_359
这个家伙很懒,什么也没留下!
PHP1.CN | 中国最专业的PHP中文社区 | DevBox开发工具箱 | json解析格式化 |PHP资讯 | PHP教程 | 数据库技术 | 服务器技术 | 前端开发技术 | PHP框架 | 开发工具 | 在线工具
Copyright © 1998 - 2020 PHP1.CN. All Rights Reserved | 京公网安备 11010802041100号 | 京ICP备19059560号-4 | PHP1.CN 第一PHP社区 版权所有