Move Legacy Content to Content System




 
Move Legacy Content to Content System

Date Published: May 24,2018 Category: Product:Data_Management_and_Integrations_Learn; Version:Learn_9_1_Q2_2017_3200_0_0,Learn_9_1_Q4_2017,Learn_9_1_Q2_2018,Learn_9_1_Q4_2016_3100_0_0_rel_107_401e,Learn_October_2014_9_1_201410_160373,SaaS,Learn_9_1_Q4_2015_9_1_201510_1171621,Learn_9_1_Q2_2016_3000_1_0_rel_52_991d   Article No.: 000071123

Product: Blackboard Learn

Document Type: Administrator Documentation

Document Summary: These instructions will guide you through the process to move all legacy content to our embedded file system (Xythos) on a Self-Hosted system.  This is one of the pre-requisite steps for migrating to Learn SaaS via full-system migration.  Please see Prepare to Migration to Blackboard Learn SaaS for more information.
Information:

Pre-requisites:

  1. Submit a support case on Behind the Blackboard to obtain the migration-tools.zip needed to run these steps.
  2. Test running the Move Legacy Content tool on a staging or other non-production system and verify content links after running the tool.
  3. Always take a full system backup (application and database) prior to running the tool on Production.
  4. Install Java 1.8 on the application server if not already present (this does not require changing the Java version used by Blackboard Learn itself).
  5. Confirm disk space availability. Ensure that at least double the current disk usage allocated to the \blackboard\content\vi\[bb_bb60\BBLEARN]\1\courses folders is freely available
  6. Consider adding more RAM and CPU. If you can add extra resources to the application server you are running the tool on, this will help it run more quickly – for example 18GB RAM and 6 virtual CPUs.
Please see Prepare to Migrate to Blackboard Learn SaaS
 

Move Legacy Content 

It is critical that an institution is using one of the following versions before running this utility. Releases that support the Move Legacy Content tool are as follows:
  • 9.1 October 2014 Cumulative Update 6 (9.1.201410.160373) and higher
  • 9.1 Q4 2015 Cumulative Update 3 (9.1.201510.1171621) and higher
  • 9.1 Q2 2016 Cumulative Update 4 (3000.1.4-rel.183+298455c) and higher
  • 9.1 Q4 2016 Cumulative Update 1 (3100.0.1-rel.117+6ef1843) and higher
Instructions:
  1. Ensure migration folder (and contents) is owned by bbuser account
  2. Edit the following depending on whether you’re running Linux or Windows:
    • (LINUX):  Edit the env.sh file in migration root directory to point to Java 1.8 on this machine
    • (WINDOWS):  Edit the 4 CMD files (audit.cmd, auditcleanup.cmd, movecontent.cmd, and s3upload.cmd) and change the following:
      • JAVA_HOME: set to the Java path to the Java 1.8 directory where the \bin directory is located (e.g. C:\Java\jdk1.8.0_11)
      • BBDIR: set to the location of the Blackboard install directory (e.g. E:\blackboard)
  3. Edit the -Xmx setting in movecontent.sh (Linux) or movecontent.cmd (Windows) so the tool can use as much RAM as is available on the server. Leave a buffer of 2G to be safe. If the server is not in production (i.e. it is not actively being used), you should stop blackboard services to free up more memory
    • Linux: to determine the available RAM, use free -g or top
    • Windows: to determine the available RAM, see Task Manager > Performance > Physical Memory > Free
  4. Edit /lib/migration.properties:
    1. Set “source.directory” to the same directory as the parameter “bbconfig.cs.external.storage.location” in the file blackboard\config\bb-config.properties (e.g. E:\blackboard\content\storage or /usr/local/blackboard/storage)
    2. move.thread.count (default is 25) - set this value so that there are 300m of RAM in the JVM (see the Xmx setting in previous step) per thread i.e. divide the Xmx setting by 300 to determine the move.thread.count. For example, if your Xmx setting is 6000m you should have a move.thread.count of no more than 20. Other considerations:
      • Make sure that move.thread.count is lower than the max-pool-size setting for the data-store named oracle-template (Linux) or mssql-template (Windows) in \blackboard\config\bb-datastores-standalone.xml. If it is higher, you can edit \blackboard\config\bb-datastores-standalone.xml to increase the max-pool-size setting (does not require a restart or a PushConfigUpdate). We recommend that the max_pool_size be at least 10 greater than move.thread.count.
Use a lower move.thread.count if this server is in production to lower the CPU impact on Blackboard processes. In this case, we recommend you start by dividing the Xmx setting by 600 (half the number of threads).
  1. batch.size (default is 50) - set this value to limit the amount of time a query remains open for the duration of the process.
  2. smtp.host - set this to the same value as the bbconfig.smtpserver.hostname property in your bb-config.properties file.
  3. smtp.notify - set this to e-mail address that should receive status messages.
  4. smtp.subject.header - set this to an easily identifiable attribute to differentiate ongoing notifications from one another.
  5. log.file - set this to a file location to write results from the tool. User bbuser must have write permission to this file location.
On Windows, this must be a file name (not path).
  1. Execute audit to identify full listing of files earmarking those ready for removal. (Run in report mode).
    • Linux: ./audit.sh --blackboard.content.home=/usr/local/blackboard/content --log.file=logs/pre-audit.log --mode=report --extended –detailed
    • Windows: audit --blackboard.content.home=\blackboard\content --log.file=c:\temp\pre-audit.log --mode=report --extended --detailed
  2. Execute movecontent script from migration archive to do the actual move.  This will run in the background on Linux, and foreground on Windows (for Linux, you can use movecontent_fg.sh for foreground if desired). The length of the process will vary based on number of courses and amount of legacy content within the course. To monitor the job review the log file in blackboard\logs\move-course-content\move-course-content-log.txt or that which appears in the log.file setting.
    • Linux: ./movecontent.sh --all
    • Windows: movecontent --all
  3. You can also monitor log.file location to identify success/failure of operation per course.
  4. Execute audit to provide final listing of files while also purging orphaned files. (Run in execute mode)
    • Linux: ./auditcleanup.sh --blackboard.content.home=/usr/local/blackboard/content --log.file=logs/pre-audit.log --mode=execute --extended –detailed
    • Windows: auditcleanup --blackboard.content.home=C:\blackboard\content --log.file=c:\temp\pre-audit.log --mode=report --extended –detailed
  5. Verify content links by testing a range of courses, especially content added by third party building blocks and content embedded in assessments

Verification

  1. If you configured the tool to send status e-mails, look for an e-mail that says MoveInstallationApplication complete
  2. Compare the total records processed count in the final email to the known number of course_main records in the database. The numbers should match or be fairly close to each other
  3. Ensure that there are no system failure messages in /blackboard/logs/move-course-content/move-course-content-log.txt
  4. Review log.file target to corroborate if failures blocked process completion
You can re-run the tool multiple times if needed.


Troubleshooting


If you get intermittent non-fatal java.lang.OutOfMemoryError in the log file, this indicates you have some large files that your JVM’s RAM can’t accommodate.

Possible Solutions:
  • Decrease the move.thread.count in /lib/migration.properties or
  • Increase the -Xmx size in movecoursecontent.sh. We recommend that you allocate at least 300m per thread. If move.thread.count is 20, your -Xmx should be 6000m
  • After changing the parameters, you can run movecoursecontent again. It will only try to move the files that have not moved yet

If you get a fatal error saying failed; error='Cannot allocate memory', your -Xmx setting is greater than the available RAM on the box. Either edit movecontent.sh (or movecontent_fg.sh) to decrease the -Xmx setting, or increase the amount of available RAM on the box by stopping other processes or reconfiguring the box.

You can monitor the MoveCourseContent log, located in blackboard/logs/move-course-content/move-course-content-log.txt, to follow progress and for warnings specific to moving content within a course. Search for WARNING to find these errors.