Advanced Unix Data Tools & Techniques

4 Day Course
Hands On
Code UXADV-1

This course has been retired. Please view currently available Unix Training Courses.


Hide all

Basics of UNIX Scripts and Data Streams (6 topics)

  • File structure
  • Shell and data streams
  • Redirection and piping
  • File descriptors
  • Creating and running scripts
  • 'Hash-bang' header

Shell Tips and Tricks (10 topics)

  • Using the shell efficiently
  • Data extraction
  • IFS and set
  • IFS and read
  • Reading into an array
  • Dealing with sub-strings
  • Process substitution
  • Output generation
  • printf
  • Why use sed, awk, or Perl?

Basic Regular Expressions (4 topics)

  • Purpose of RE
  • Using grep to illustrate basic RE syntax
  • Anchors
  • Quantifiers

Extended Regular Expressions (4 topics)

  • Typed wildcards
  • Matching alternatives
  • Minimal and 'greedy' match
  • Sub-patterns

Basic sed operation (9 topics)

  • Principles of sed stream processing
  • Invoking sed from the command line
  • Invoking sed in a shell script
  • Writing a sed script
  • Sed's command line options
  • Standard sed's editing commands
  • Line addressing
  • RE in line addressing and searches
  • Reading and writing files

Advanced sed operation (4 topics)

  • Multi-line pattern space
  • Creating loops
  • Hold space
  • Labels and branching

Basic awk operation (10 topics)

  • Principles of awk stream processing
  • Writing and invoking awk script
  • Awk's command line options
  • Standard sed's editing commands
  • Records and Fields
  • Expressions
  • Variables and constants
  • Operators
  • Line addressing
  • RE in line addressing and searches

Advanced awk operation (5 topics)

  • Passing Positional parameters into awk script
  • Working with shell variables
  • Conditional statements
  • Lopping statements, Array and associative array
  • Functions

Introduction to Perl (5 topics)

  • Documentation
  • Running Perl scripts
  • basic syntax
  • Perl variables and literals
  • Interpolation

Perl Expressions and Operators (5 topics)

  • Scalar operators
  • Context and type conversions
  • Scalar functions
  • Converting between lists and strings
  • Hash and array functions

Perl Decision Making (6 topics)

  • Basic conditional statements
  • Boolean and logical operators
  • File condition functions
  • Regular expressions as conditions
  • Loops
  • Loops for reading files

Advanced Perl (4 topics)

  • Writing to files
  • Subroutines
  • Using Modules
  • Advanced regular Expressions

Appendices (5 topics)

  • Supporting tools
  • SCCS
  • RCS
  • Introduction to Python
  • Bash regular expressions


Delegates must have good experience of using Unix systems, and ideally of developing automated solutions using shell scripting. Familiarity with basic data manipulation tools (e.g. grep), basic regular expressions, and principles of scripting is essential. Typically, the prerequisites can be met by attending the UNIX Fundamentals course followed by the Mastering Unix Shell Scripting course.

Additional Learning

The courses below may help you meet the knowledge level required to take this course. If you are unsure please ask a training advisor .