Skip to content

Optimize multi-index and FTS index insert/update on SQL Providers#64

Merged
berickson1 merged 3 commits into
masterfrom
brericks/batchIndex
Oct 1, 2018
Merged

Optimize multi-index and FTS index insert/update on SQL Providers#64
berickson1 merged 3 commits into
masterfrom
brericks/batchIndex

Conversation

@berickson1
Copy link
Copy Markdown
Collaborator

We can do the insert and deletes of multi-key indexes in one pass instead of executing multiple queries

Comment thread src/SqlProviderBase.ts Outdated
We can do the insert and deletes of multi-key indexes in one pass instead of executing multiple queries
Comment thread src/SqlProviderBase.ts Outdated
Comment thread src/SqlProviderBase.ts
Comment thread src/SqlProviderBase.ts Outdated
Comment thread src/SqlProviderBase.ts Outdated
// data contains all the input parameters
for (let i = 0; i < (data.length / insertParamCount); i += itemPageSize) {
const thisPageCount = Math.min(itemPageSize, (data.length / insertParamCount)) - i;
const qmarksValues = _.fill(new Array(thisPageCount), generateParamPlaceholder(insertParamCount));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: The same _.repeat idea could be used here instead of _.fill, Array, and .join.

_.repeat('(' + generateParamPlaceholder(insertParamCount) + '),', thisPageCount).slice(0, -1);

Perhaps this could be extracted:

repeatWithSeparator(str: string, n: number, sep: string) {
    return _.repeat(str + sep, n).slice(0, -sep.length);
}

@berickson1 berickson1 merged commit da5a95a into master Oct 1, 2018
@berickson1 berickson1 deleted the brericks/batchIndex branch October 1, 2018 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants