Bareos Main Reference and Documentation

for Bareos version 18.4 release 18.4.1 , created Dec 11, 2018

• 1. Introduction and Tutorial
  • 1.1. What is Bareos?
    • 1.1.1. History
    • 1.1.2. Who Needs Bareos?
    • 1.1.3. Bareos Components or Services
    • 1.1.4. Bareos Version Numbers and Releases
    • 1.1.5. Bareos Packages
    • 1.1.6. Quick Start
    • 1.1.7. Terminology
    • 1.1.8. What Bareos is Not
    • 1.1.9. Interactions Between the Bareos Services
  • 1.2. The Current State of Bareos
    • 1.2.1. What is Implemented
    • 1.2.2. Advantages Over Other Backup Programs
    • 1.2.3. Current Implementation Restrictions
    • 1.2.4. Design Limitations or Restrictions
    • 1.2.5. Items to Note
  • 1.3. Installing Bareos
    • 1.3.1. Decide about the Bareos release to use
    • 1.3.2. Decide about the Database Backend
    • 1.3.3. Install the Bareos Software Packages
    • 1.3.4. Prepare Bareos database
    • 1.3.5. Start the daemons
  • 1.4. Installing Bareos Webui
    • 1.4.1. Features
    • 1.4.2. System Requirements
    • 1.4.3. Installation
    • 1.4.4. Upgrade from 15.2 to 16.2
    • 1.4.5. Additional information
  • 1.5. Updating Bareos
    • 1.5.1. Updating the configuration files
    • 1.5.2. Updating the database scheme
  • 1.6. Getting Started with Bareos
    • 1.6.1. Understanding Jobs and Schedules
    • 1.6.2. Understanding Pools, Volumes and Labels
    • 1.6.3. Setting Up Bareos Configuration Files
    • 1.6.4. Testing your Configuration Files
  • 1.7. Tutorial
    • 1.7.1. Starting the Database
    • 1.7.2. Installing Bareos
    • 1.7.3. Starting the Daemons
    • 1.7.4. Using the Director to Query and Start Jobs
    • 1.7.5. Running a Job
    • 1.7.6. Restoring Your Files
    • 1.7.7. Quitting the Console Program
    • 1.7.8. Adding a Client
    • 1.7.9. Patience When Starting Daemons or Mounting Blank Tapes
    • 1.7.10. Pools
    • 1.7.11. Other Useful Console Commands
  • 1.8. Critical Items to Implement Before Production
    • 1.8.1. Critical Items
    • 1.8.2. Recommended Items
• 2. Configuration
  • 2.1. Customizing the Configuration
    • 2.1.1. Configuration Path Layout
    • 2.1.2. Configuration File Format
    • 2.1.3. Resource
    • 2.1.4. Names, Passwords and Authorization
  • 2.2. Director Configuration
    • 2.2.1. Director Resource
    • 2.2.2. Job Resource
    • 2.2.3. JobDefs Resource
    • 2.2.4. Schedule Resource
    • 2.2.5. FileSet Resource
    • 2.2.6. Client Resource
    • 2.2.7. Storage Resource
    • 2.2.8. Pool Resource
    • 2.2.9. Catalog Resource
    • 2.2.10. Messages Resource
    • 2.2.11. Console Resource
    • 2.2.12. Profile Resource
    • 2.2.13. Counter Resource
    • 2.2.14. Example Director Configuration File
  • 2.3. Storage Daemon Configuration
    • 2.3.1. Storage Resource
    • 2.3.2. Director Resource
    • 2.3.3. NDMP Resource
    • 2.3.4. Device Resource
    • 2.3.5. Messages Resource
    • 2.3.6. Example Storage Daemon Configuration File
  • 2.4. Client/File Daemon Configuration
    • 2.4.1. Client Resource
    • 2.4.2. Director Resource
    • 2.4.3. Messages Resource
    • 2.4.4. Example Client Configuration File
  • 2.5. Messages Resource
    • 2.5.1. Message Types
  • 2.6. Console Configuration
    • 2.6.1. Director Resource
    • 2.6.2. Console Resource
    • 2.6.3. Example Console Configuration File
  • 2.7. Monitor Configuration
    • 2.7.1. Monitor Resource
    • 2.7.2. Director Resource
    • 2.7.3. Client Resource
    • 2.7.4. Storage Resource
    • 2.7.5. Tray Monitor
• 3. Task and Concepts
  • 3.1. Bareos Console
    • 3.1.1. Console Configuration
    • 3.1.2. Running the Console Program
    • 3.1.3. Console Keywords
    • 3.1.4. Console Commands
    • 3.1.5. Adding Volumes to a Pool
  • 3.2. The Restore Command
    • 3.2.1. General
    • 3.2.2. The Restore Command
    • 3.2.3. Selecting Files by Filename
    • 3.2.4. Replace Options
    • 3.2.5. Command Line Arguments
    • 3.2.6. Using File Relocation
    • 3.2.7. Restoring Directory Attributes
    • 3.2.8. Restoring on Windows
    • 3.2.9. Restore Errors
    • 3.2.10. Example Restore Job Resource
    • 3.2.11. File Selection Commands
  • 3.3. Volume Management
    • 3.3.1. Key Concepts and Resource Records
    • 3.3.2. Concurrent Disk Jobs
    • 3.3.3. Automatic Volume Recycling
  • 3.4. Automated Disk Backup
    • 3.4.1. Overall Design
    • 3.4.2. Configuration Files
  • 3.5. Autochanger Support
    • 3.5.1. Knowing What SCSI Devices You Have
    • 3.5.2. Slots
    • 3.5.3. Multiple Devices
    • 3.5.4. Device Configuration Records
    • 3.5.5. Specifying Slots When Labeling
    • 3.5.6. Changing Cartridges
    • 3.5.7. Dealing with Multiple Magazines
    • 3.5.8. Update Slots Command
    • 3.5.9. Using the Autochanger
    • 3.5.10. Barcode Support
    • 3.5.11. Use bconsole to display Autochanger content
    • 3.5.12. Bareos Autochanger Interface
    • 3.5.13. Tapespeed and blocksizes
    • 3.5.14. Tape Drive Cleaning
  • 3.6. Using Tape Drives without Autochanger
    • 3.6.1. Simple One Tape Backup
    • 3.6.2. Manually Changing Tapes
    • 3.6.3. Daily Tape Rotation
  • 3.7. Data Spooling
    • 3.7.1. Data Spooling Directives
  • 3.8. Migration and Copy
    • 3.8.1. Important Migration Considerations
    • 3.8.2. Configure Copy or Migration Jobs
  • 3.9. Always Incremental Backup Scheme
    • 3.9.1. Conventional Backup Scheme Drawbacks
    • 3.9.2. Always Incremental Concept
    • 3.9.3. How to configure in Bareos
    • 3.9.4. How it works
    • 3.9.5. Enhancements for the Always Incremental Backup Scheme
    • 3.9.6. Long Term Storage of Always Incremental Jobs
  • 3.10. How to manually transfer data/volumes
    • 3.10.1. Import Data from a Remote Storage Daemon
    • 3.10.2. Import Data from a Independent Remote Full Bareos Installation
  • 3.11. File Deduplication using Base Jobs
  • 3.12. Plugins
    • 3.12.1. File Daemon Plugins
    • 3.12.2. Storage Daemon Plugins
    • 3.12.3. Director Plugins
  • 3.13. The Windows Version of Bareos
    • 3.13.1. Windows Installation
    • 3.13.2. Dealing with Windows Problems
    • 3.13.3. Windows Compatibility Considerations
    • 3.13.4. Volume Shadow Copy Service (VSS)
    • 3.13.5. Windows Firewalls
    • 3.13.6. Windows Restore Problems
    • 3.13.7. Windows Backup Problems
    • 3.13.8. Windows Ownership and Permissions Problems
    • 3.13.9. Advanced Windows Configuration
  • 3.14. Network setup
    • 3.14.1. Client Initiated Connection
    • 3.14.2. Passive Clients
    • 3.14.3. Using different IP Adresses for SD – FD Communication
  • 3.15. Transport Encryption
    • 3.15.1. TLS Configuration Directives
    • 3.15.2. Getting TLS Certificates
    • 3.15.3. Example TLS Configuration Files
  • 3.16. Data Encryption
    • 3.16.1. Encryption Technical Details
    • 3.16.2. Generating Private/Public Encryption Keys
    • 3.16.3. Example Data Encryption Configurations (bareos-fd.conf)
    • 3.16.4. Decrypting with a Master Key
  • 3.17. NDMP Backups with Bareos
    • 3.17.1. NDMP Basics
    • 3.17.2. NDMP Backup in Bareos
    • 3.17.3. NDMP_BAREOS
    • 3.17.4. NDMP_NATIVE
    • 3.17.5. NDMP Common
  • 3.18. Catalog Maintenance
    • 3.18.1. Catalog Database
    • 3.18.2. Retention Periods
    • 3.18.3. PostgreSQL
    • 3.18.4. MySQL/MariaDB
    • 3.18.5. Backing Up Your Bareos Database
  • 3.19. Bareos Security Issues
    • 3.19.1. Configuring and Testing TCP Wrappers
    • 3.19.2. Secure Erase Command
• 4. Appendix
  • 4.1. System Requirements
  • 4.2. Operating Systems
    • 4.2.1. Packages for the different Linux platforms
    • 4.2.2. Debian.org / Ubuntu Universe
    • 4.2.3. Mac OS X
  • 4.3. Bareos Programs
    • 4.3.1. Bareos Daemons
    • 4.3.2. Interactive Programs
    • 4.3.3. Volume Utility Commands
    • 4.3.4. Other Programs
  • 4.4. The Bootstrap File
    • 4.4.1. Bootstrap File Format
    • 4.4.2. Automatic Generation of Bootstrap Files
    • 4.4.3. Bootstrap for bscan
    • 4.4.4. Bootstrap Example
  • 4.5. Verify File Integrity with Bareos
    • 4.5.1. The Details
    • 4.5.2. Running the Verify
    • 4.5.3. What To Do When Differences Are Found
    • 4.5.4. A Verify Configuration Example
  • 4.6. Backward Compatibility
    • 4.6.1. Tape Formats
    • 4.6.2. Compatibility between Bareos and Bacula
  • 4.7. Catalog Tables
    • 4.7.1. Job
  • 4.8. Howtos
    • 4.8.1. Use a dummy device to test the backup
    • 4.8.2. Backup Of Third Party Databases
  • 4.9. Disaster Recovery Using Bareos
    • 4.9.1. General
    • 4.9.2. Steps to Take Before Disaster Strikes
    • 4.9.3. Bare Metal Recovery of Bareos Clients
    • 4.9.4. Restoring a Bareos Server
  • 4.10. Troubleshooting
    • 4.10.1. Debug Messages
    • 4.10.2. Client Access Problems
    • 4.10.3. Concurrent Jobs
    • 4.10.4. Media VolWrites: integer out of range
    • 4.10.5. Tape Drive
    • 4.10.6. Autochanger
    • 4.10.7. Restore
  • 4.11. Debugging
    • 4.11.1. Traceback
    • 4.11.2. Testing The Traceback
    • 4.11.3. Manually Running Bareos Under The Debugger
  • 4.12. Release Notes
  • 4.13. Bareos Copyright, Trademark and Licenses
    • 4.13.1. Licenses Overview
• 5. Bareos Developer Guide
  • 5.1. Bareos Developer Notes
    • 5.1.1. History
    • 5.1.2. Contributions
    • 5.1.3. Patches
    • 5.1.4. Compiling
    • 5.1.5. Debugging
    • 5.1.6. Using a Debugger
    • 5.1.7. Memory Leaks
    • 5.1.8. When Implementing Incomplete Code
    • 5.1.9. Header Files
    • 5.1.10. Programming Standards
    • 5.1.11. Do Not Use
    • 5.1.12. Avoid if Possible
    • 5.1.13. Do Use Whenever Possible
    • 5.1.14. Indenting Standards
    • 5.1.15. Tabbing
    • 5.1.16. Don’ts
  • 5.2. Message Classes
    • 5.2.1. Memory Messages
    • 5.2.2. Debug Messages
    • 5.2.3. Job Messages
    • 5.2.4. Queued Job Messages
    • 5.2.5. Error Messages
  • 5.3. Bareos FD Plugin API
    • 5.3.1. Normal vs Command vs Options Plugins
    • 5.3.2. Loading Plugins
    • 5.3.3. loadPlugin
    • 5.3.4. Plugin Entry Points
    • 5.3.5. Bareos Plugin Entrypoints
    • 5.3.6. Building Bareos Plugins
  • 5.4. Platform Support
    • 5.4.1. General
    • 5.4.2. Requirements to become a Supported Platform
  • 5.5. Daemon Protocol
    • 5.5.1. General
    • 5.5.2. Low Level Network Protocol
    • 5.5.3. General Daemon Protocol
    • 5.5.4. The Protocol Used Between the Director and the Storage Daemon
    • 5.5.5. The Protocol Used Between the Director and the File Daemon
    • 5.5.6. The Save Protocol Between the File Daemon and the Storage Daemon
  • 5.6. File Services Daemon
    • 5.6.1. Commands Received from the Director for a Backup
    • 5.6.2. Commands Received from the Director for a Restore
  • 5.7. Storage Daemon Design
    • 5.7.1. SD Design Introduction
    • 5.7.2. SD Development Outline
    • 5.7.3. SD Connections and Sessions
    • 5.7.4. SD Data Structures
  • 5.8. Catalog Services
    • 5.8.1. General
    • 5.8.2. Sequence of Creation of Records for a Save Job
    • 5.8.3. Database Tables
  • 5.9. Storage Media Output Format
    • 5.9.1. General
    • 5.9.2. Definitions
    • 5.9.3. Overall Format
    • 5.9.4. Serialization
    • 5.9.5. Block Header
    • 5.9.6. Record Header
    • 5.9.7. Volume Label Format
    • 5.9.8. Session Label
    • 5.9.9. Overall Storage Format
    • 5.9.10. Examine Volumes
    • 5.9.11. Unix File Attributes
    • 5.9.12. Old Deprecated Tape Format
  • 5.10. Bareos Porting Notes
    • 5.10.1. Porting Requirements
    • 5.10.2. Steps to Take for Porting
  • 5.11. API
    • 5.11.1. General
    • 5.11.2. dot commands
    • 5.11.3. API Modes
    • 5.11.4. Bvfs API
  • 5.12. TLS
    • 5.12.1. Introduction to TLS
    • 5.12.2. TLS API Implementation
    • 5.12.3. Bnet API Changes
    • 5.12.4. Authentication Negotiation
  • 5.13. Bareos Regression Testing
    • 5.13.1. Setting up Regession Testing
    • 5.13.2. Running the Regression Script
    • 5.13.3. Testing a Binary Installation
    • 5.13.4. Running a Single Test
    • 5.13.5. Writing a Regression Test
  • 5.14. Bareos Memory Management
    • 5.14.1. General
  • 5.15. TCP/IP Network Protocol
    • 5.15.1. General
    • 5.15.2. bnet and Threads
    • 5.15.3. bnet_open
    • 5.15.4. bnet_send
    • 5.15.5. bnet_fsend
    • 5.15.6. is_bnet_error
    • 5.15.7. is_bnet_stop
    • 5.15.8. bnet_recv
    • 5.15.9. bnet_sig
    • 5.15.10. bnet_strerror
    • 5.15.11. bnet_close
    • 5.15.12. Becoming a Server
    • 5.15.13. Higher Level Conventions
  • 5.16. Director Console Output
    • 5.16.1. object_key_value
    • 5.16.2. decoration
    • 5.16.3. messages
    • 5.16.4. Objects and Arrays
    • 5.16.5. Example
    • 5.16.6. Example with 3 level structure
  • 5.17. Smart Memory Allocation
    • 5.17.1. Installing SMARTALLOC
    • 5.17.2. Squelching a SMARTALLOC
    • 5.17.3. Living with Libraries
    • 5.17.4. SMARTALLOC Details
    • 5.17.5. When SMARTALLOC is Disabled
    • 5.17.6. The alloc() Function
    • 5.17.7. Overlays and Underhandedness
    • 5.17.8. Test and Demonstration Program
    • 5.17.9. Invitation to the Hack
  • 5.18. GNU Free Documentation License

Indices and tables

• Index