saint/discourse-import_scripts
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

v0.39.7 Fixes/tweaks in calculate_dynamic_pool_size

Browse Source
This commit is contained in:
David Sainty 2024-08-17 03:06:07 +10:00
parent 77930217df
commit 524fae9283
1 changed files with 7 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
gossamer_forums.rb
View File

@ -1,7 +1,7 @@
# Federated Computer, Inc. # Federated Computer, Inc.
# David Sainty <saint@federated.computer> 2024 A.D. # David Sainty <saint@federated.computer> 2024 A.D.
# Gossamer Threads to Discourse -- Migration-Import Script # Gossamer Threads to Discourse -- Migration-Import Script
# v0.39.6 Fixes/tweaks in calculate_dynamic_pool_size # v0.39.7 Fixes/tweaks in calculate_dynamic_pool_size
require 'mysql2' require 'mysql2'
require 'open-uri' require 'open-uri'
@ -1058,9 +1058,13 @@ class GossamerForumsImporter < ImportScripts::Base
def threaded_topic_import def threaded_topic_import
# Get list of TOPICS / OP posts, i.e. post ids that have no parent / root id - SELECT post_id FROM gforum_Post WHERE post_root_id = 0; # Get list of TOPICS / OP posts, i.e. post ids that have no parent / root id - SELECT post_id FROM gforum_Post WHERE post_root_id = 0;
# The query selects post_ids from gforum_Post where post_root_id is 0, meaning these posts are the topic starters (OPs). # The query selects post_ids from gforum_Post where post_root_id is 0, meaning these posts are the topic starters (OPs).
# It also ensures that we only process posts with a post_id greater than the last processed one, allowing for resumption. # Execute the query and fetch the result
parent_post_ids = execute_query("SELECT post_id FROM gforum_Post WHERE post_root_id = 0 ORDER BY post_id ASC") result = execute_query("SELECT post_id FROM gforum_Post WHERE post_root_id = 0 ORDER BY post_id ASC")
# Convert the result set to an array of post_ids
parent_post_ids = result.map { |row| row['post_id'] }
parent_post_count = parent_post_ids.count parent_post_count = parent_post_ids.count
batch_size = 100 # Set our batch size for number of posts to import in a single batch batch_size = 100 # Set our batch size for number of posts to import in a single batch