<!--{{{-->
<link rel='alternate' type='application/rss+xml' title='RSS' href='index.xml' />
<!--}}}-->
Background: #fff
Foreground: #000
PrimaryPale: #8cf
PrimaryLight: #18f
PrimaryMid: #04b
PrimaryDark: #014
SecondaryPale: #ffc
SecondaryLight: #fe8
SecondaryMid: #db4
SecondaryDark: #841
TertiaryPale: #eee
TertiaryLight: #ccc
TertiaryMid: #999
TertiaryDark: #666
Error: #f88
/*{{{*/
body {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}

a {color:[[ColorPalette::PrimaryMid]];}
a:hover {background-color:[[ColorPalette::PrimaryMid]]; color:[[ColorPalette::Background]];}
a img {border:0;}

h1,h2,h3,h4,h5,h6 {color:[[ColorPalette::SecondaryDark]]; background:transparent;}
h1 {border-bottom:2px solid [[ColorPalette::TertiaryLight]];}
h2,h3 {border-bottom:1px solid [[ColorPalette::TertiaryLight]];}

.button {color:[[ColorPalette::PrimaryDark]]; border:1px solid [[ColorPalette::Background]];}
.button:hover {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::SecondaryLight]]; border-color:[[ColorPalette::SecondaryMid]];}
.button:active {color:[[ColorPalette::Background]]; background:[[ColorPalette::SecondaryMid]]; border:1px solid [[ColorPalette::SecondaryDark]];}

.header {background:[[ColorPalette::PrimaryMid]];}
.headerShadow {color:[[ColorPalette::Foreground]];}
.headerShadow a {font-weight:normal; color:[[ColorPalette::Foreground]];}
.headerForeground {color:[[ColorPalette::Background]];}
.headerForeground a {font-weight:normal; color:[[ColorPalette::PrimaryPale]];}

.tabSelected{color:[[ColorPalette::PrimaryDark]];
	background:[[ColorPalette::TertiaryPale]];
	border-left:1px solid [[ColorPalette::TertiaryLight]];
	border-top:1px solid [[ColorPalette::TertiaryLight]];
	border-right:1px solid [[ColorPalette::TertiaryLight]];
}
.tabUnselected {color:[[ColorPalette::Background]]; background:[[ColorPalette::TertiaryMid]];}
.tabContents {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::TertiaryPale]]; border:1px solid [[ColorPalette::TertiaryLight]];}
.tabContents .button {border:0;}

#sidebar {}
#sidebarOptions input {border:1px solid [[ColorPalette::PrimaryMid]];}
#sidebarOptions .sliderPanel {background:[[ColorPalette::PrimaryPale]];}
#sidebarOptions .sliderPanel a {border:none;color:[[ColorPalette::PrimaryMid]];}
#sidebarOptions .sliderPanel a:hover {color:[[ColorPalette::Background]]; background:[[ColorPalette::PrimaryMid]];}
#sidebarOptions .sliderPanel a:active {color:[[ColorPalette::PrimaryMid]]; background:[[ColorPalette::Background]];}

.wizard {background:[[ColorPalette::PrimaryPale]]; border:1px solid [[ColorPalette::PrimaryMid]];}
.wizard h1 {color:[[ColorPalette::PrimaryDark]]; border:none;}
.wizard h2 {color:[[ColorPalette::Foreground]]; border:none;}
.wizardStep {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];
	border:1px solid [[ColorPalette::PrimaryMid]];}
.wizardStep.wizardStepDone {background:[[ColorPalette::TertiaryLight]];}
.wizardFooter {background:[[ColorPalette::PrimaryPale]];}
.wizardFooter .status {background:[[ColorPalette::PrimaryDark]]; color:[[ColorPalette::Background]];}
.wizard .button {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::SecondaryLight]]; border: 1px solid;
	border-color:[[ColorPalette::SecondaryPale]] [[ColorPalette::SecondaryDark]] [[ColorPalette::SecondaryDark]] [[ColorPalette::SecondaryPale]];}
.wizard .button:hover {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::Background]];}
.wizard .button:active {color:[[ColorPalette::Background]]; background:[[ColorPalette::Foreground]]; border: 1px solid;
	border-color:[[ColorPalette::PrimaryDark]] [[ColorPalette::PrimaryPale]] [[ColorPalette::PrimaryPale]] [[ColorPalette::PrimaryDark]];}

.wizard .notChanged {background:transparent;}
.wizard .changedLocally {background:#80ff80;}
.wizard .changedServer {background:#8080ff;}
.wizard .changedBoth {background:#ff8080;}
.wizard .notFound {background:#ffff80;}
.wizard .putToServer {background:#ff80ff;}
.wizard .gotFromServer {background:#80ffff;}

#messageArea {border:1px solid [[ColorPalette::SecondaryMid]]; background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]];}
#messageArea .button {color:[[ColorPalette::PrimaryMid]]; background:[[ColorPalette::SecondaryPale]]; border:none;}

.popupTiddler {background:[[ColorPalette::TertiaryPale]]; border:2px solid [[ColorPalette::TertiaryMid]];}

.popup {background:[[ColorPalette::TertiaryPale]]; color:[[ColorPalette::TertiaryDark]]; border-left:1px solid [[ColorPalette::TertiaryMid]]; border-top:1px solid [[ColorPalette::TertiaryMid]]; border-right:2px solid [[ColorPalette::TertiaryDark]]; border-bottom:2px solid [[ColorPalette::TertiaryDark]];}
.popup hr {color:[[ColorPalette::PrimaryDark]]; background:[[ColorPalette::PrimaryDark]]; border-bottom:1px;}
.popup li.disabled {color:[[ColorPalette::TertiaryMid]];}
.popup li a, .popup li a:visited {color:[[ColorPalette::Foreground]]; border: none;}
.popup li a:hover {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; border: none;}
.popup li a:active {background:[[ColorPalette::SecondaryPale]]; color:[[ColorPalette::Foreground]]; border: none;}
.popupHighlight {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}
.listBreak div {border-bottom:1px solid [[ColorPalette::TertiaryDark]];}

.tiddler .defaultCommand {font-weight:bold;}

.shadow .title {color:[[ColorPalette::TertiaryDark]];}

.title {color:[[ColorPalette::SecondaryDark]];}
.subtitle {color:[[ColorPalette::TertiaryDark]];}

.toolbar {color:[[ColorPalette::PrimaryMid]];}
.toolbar a {color:[[ColorPalette::TertiaryLight]];}
.selected .toolbar a {color:[[ColorPalette::TertiaryMid]];}
.selected .toolbar a:hover {color:[[ColorPalette::Foreground]];}

.tagging, .tagged {border:1px solid [[ColorPalette::TertiaryPale]]; background-color:[[ColorPalette::TertiaryPale]];}
.selected .tagging, .selected .tagged {background-color:[[ColorPalette::TertiaryLight]]; border:1px solid [[ColorPalette::TertiaryMid]];}
.tagging .listTitle, .tagged .listTitle {color:[[ColorPalette::PrimaryDark]];}
.tagging .button, .tagged .button {border:none;}

.footer {color:[[ColorPalette::TertiaryLight]];}
.selected .footer {color:[[ColorPalette::TertiaryMid]];}

.sparkline {background:[[ColorPalette::PrimaryPale]]; border:0;}
.sparktick {background:[[ColorPalette::PrimaryDark]];}

.error, .errorButton {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::Error]];}
.warning {color:[[ColorPalette::Foreground]]; background:[[ColorPalette::SecondaryPale]];}
.lowlight {background:[[ColorPalette::TertiaryLight]];}

.zoomer {background:none; color:[[ColorPalette::TertiaryMid]]; border:3px solid [[ColorPalette::TertiaryMid]];}

.imageLink, #displayArea .imageLink {background:transparent;}

.annotation {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; border:2px solid [[ColorPalette::SecondaryMid]];}

.viewer .listTitle {list-style-type:none; margin-left:-2em;}
.viewer .button {border:1px solid [[ColorPalette::SecondaryMid]];}
.viewer blockquote {border-left:3px solid [[ColorPalette::TertiaryDark]];}

.viewer table, table.twtable {border:2px solid [[ColorPalette::TertiaryDark]];}
.viewer th, .viewer thead td, .twtable th, .twtable thead td {background:[[ColorPalette::SecondaryMid]]; border:1px solid [[ColorPalette::TertiaryDark]]; color:[[ColorPalette::Background]];}
.viewer td, .viewer tr, .twtable td, .twtable tr {border:1px solid [[ColorPalette::TertiaryDark]];}

.viewer pre {border:1px solid [[ColorPalette::SecondaryLight]]; background:[[ColorPalette::SecondaryPale]];}
.viewer code {color:[[ColorPalette::SecondaryDark]];}
.viewer hr {border:0; border-top:dashed 1px [[ColorPalette::TertiaryDark]]; color:[[ColorPalette::TertiaryDark]];}

.highlight, .marked {background:[[ColorPalette::SecondaryLight]];}

.editor input {border:1px solid [[ColorPalette::PrimaryMid]];}
.editor textarea {border:1px solid [[ColorPalette::PrimaryMid]]; width:100%;}
.editorFooter {color:[[ColorPalette::TertiaryMid]];}

#backstageArea {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::TertiaryMid]];}
#backstageArea a {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::Background]]; border:none;}
#backstageArea a:hover {background:[[ColorPalette::SecondaryLight]]; color:[[ColorPalette::Foreground]]; }
#backstageArea a.backstageSelTab {background:[[ColorPalette::Background]]; color:[[ColorPalette::Foreground]];}
#backstageButton a {background:none; color:[[ColorPalette::Background]]; border:none;}
#backstageButton a:hover {background:[[ColorPalette::Foreground]]; color:[[ColorPalette::Background]]; border:none;}
#backstagePanel {background:[[ColorPalette::Background]]; border-color: [[ColorPalette::Background]] [[ColorPalette::TertiaryDark]] [[ColorPalette::TertiaryDark]] [[ColorPalette::TertiaryDark]];}
.backstagePanelFooter .button {border:none; color:[[ColorPalette::Background]];}
.backstagePanelFooter .button:hover {color:[[ColorPalette::Foreground]];}
#backstageCloak {background:[[ColorPalette::Foreground]]; opacity:0.6; filter:'alpha(opacity=60)';}
/*}}}*/
/*{{{*/
* html .tiddler {height:1%;}

body {font-size:.75em; font-family:arial,helvetica; margin:0; padding:0;}

h1,h2,h3,h4,h5,h6 {font-weight:bold; text-decoration:none;}
h1,h2,h3 {padding-bottom:1px; margin-top:1.2em;margin-bottom:0.3em;}
h4,h5,h6 {margin-top:1em;}
h1 {font-size:1.35em;}
h2 {font-size:1.25em;}
h3 {font-size:1.1em;}
h4 {font-size:1em;}
h5 {font-size:.9em;}

hr {height:1px;}

a {text-decoration:none;}

dt {font-weight:bold;}

ol {list-style-type:decimal;}
ol ol {list-style-type:lower-alpha;}
ol ol ol {list-style-type:lower-roman;}
ol ol ol ol {list-style-type:decimal;}
ol ol ol ol ol {list-style-type:lower-alpha;}
ol ol ol ol ol ol {list-style-type:lower-roman;}
ol ol ol ol ol ol ol {list-style-type:decimal;}

.txtOptionInput {width:11em;}

#contentWrapper .chkOptionInput {border:0;}

.externalLink {text-decoration:underline;}

.indent {margin-left:3em;}
.outdent {margin-left:3em; text-indent:-3em;}
code.escaped {white-space:nowrap;}

.tiddlyLinkExisting {font-weight:bold;}
.tiddlyLinkNonExisting {font-style:italic;}

/* the 'a' is required for IE, otherwise it renders the whole tiddler in bold */
a.tiddlyLinkNonExisting.shadow {font-weight:bold;}

#mainMenu .tiddlyLinkExisting,
	#mainMenu .tiddlyLinkNonExisting,
	#sidebarTabs .tiddlyLinkNonExisting {font-weight:normal; font-style:normal;}
#sidebarTabs .tiddlyLinkExisting {font-weight:bold; font-style:normal;}

.header {position:relative;}
.header a:hover {background:transparent;}
.headerShadow {position:relative; padding:4.5em 0 1em 1em; left:-1px; top:-1px;}
.headerForeground {position:absolute; padding:4.5em 0 1em 1em; left:0px; top:0px;}

.siteTitle {font-size:3em;}
.siteSubtitle {font-size:1.2em;}

#mainMenu {position:absolute; left:0; width:10em; text-align:right; line-height:1.6em; padding:1.5em 0.5em 0.5em 0.5em; font-size:1.1em;}

#sidebar {position:absolute; right:3px; width:16em; font-size:.9em;}
#sidebarOptions {padding-top:0.3em;}
#sidebarOptions a {margin:0 0.2em; padding:0.2em 0.3em; display:block;}
#sidebarOptions input {margin:0.4em 0.5em;}
#sidebarOptions .sliderPanel {margin-left:1em; padding:0.5em; font-size:.85em;}
#sidebarOptions .sliderPanel a {font-weight:bold; display:inline; padding:0;}
#sidebarOptions .sliderPanel input {margin:0 0 0.3em 0;}
#sidebarTabs .tabContents {width:15em; overflow:hidden;}

.wizard {padding:0.1em 1em 0 2em;}
.wizard h1 {font-size:2em; font-weight:bold; background:none; padding:0; margin:0.4em 0 0.2em;}
.wizard h2 {font-size:1.2em; font-weight:bold; background:none; padding:0; margin:0.4em 0 0.2em;}
.wizardStep {padding:1em 1em 1em 1em;}
.wizard .button {margin:0.5em 0 0; font-size:1.2em;}
.wizardFooter {padding:0.8em 0.4em 0.8em 0;}
.wizardFooter .status {padding:0 0.4em; margin-left:1em;}
.wizard .button {padding:0.1em 0.2em;}

#messageArea {position:fixed; top:2em; right:0; margin:0.5em; padding:0.5em; z-index:2000; _position:absolute;}
.messageToolbar {display:block; text-align:right; padding:0.2em;}
#messageArea a {text-decoration:underline;}

.tiddlerPopupButton {padding:0.2em;}
.popupTiddler {position: absolute; z-index:300; padding:1em; margin:0;}

.popup {position:absolute; z-index:300; font-size:.9em; padding:0; list-style:none; margin:0;}
.popup .popupMessage {padding:0.4em;}
.popup hr {display:block; height:1px; width:auto; padding:0; margin:0.2em 0;}
.popup li.disabled {padding:0.4em;}
.popup li a {display:block; padding:0.4em; font-weight:normal; cursor:pointer;}
.listBreak {font-size:1px; line-height:1px;}
.listBreak div {margin:2px 0;}

.tabset {padding:1em 0 0 0.5em;}
.tab {margin:0 0 0 0.25em; padding:2px;}
.tabContents {padding:0.5em;}
.tabContents ul, .tabContents ol {margin:0; padding:0;}
.txtMainTab .tabContents li {list-style:none;}
.tabContents li.listLink { margin-left:.75em;}

#contentWrapper {display:block;}
#splashScreen {display:none;}

#displayArea {margin:1em 17em 0 14em;}

.toolbar {text-align:right; font-size:.9em;}

.tiddler {padding:1em 1em 0;}

.missing .viewer,.missing .title {font-style:italic;}

.title {font-size:1.6em; font-weight:bold;}

.missing .subtitle {display:none;}
.subtitle {font-size:1.1em;}

.tiddler .button {padding:0.2em 0.4em;}

.tagging {margin:0.5em 0.5em 0.5em 0; float:left; display:none;}
.isTag .tagging {display:block;}
.tagged {margin:0.5em; float:right;}
.tagging, .tagged {font-size:0.9em; padding:0.25em;}
.tagging ul, .tagged ul {list-style:none; margin:0.25em; padding:0;}
.tagClear {clear:both;}

.footer {font-size:.9em;}
.footer li {display:inline;}

.annotation {padding:0.5em; margin:0.5em;}

* html .viewer pre {width:99%; padding:0 0 1em 0;}
.viewer {line-height:1.4em; padding-top:0.5em;}
.viewer .button {margin:0 0.25em; padding:0 0.25em;}
.viewer blockquote {line-height:1.5em; padding-left:0.8em;margin-left:2.5em;}
.viewer ul, .viewer ol {margin-left:0.5em; padding-left:1.5em;}

.viewer table, table.twtable {border-collapse:collapse; margin:0.8em 1.0em;}
.viewer th, .viewer td, .viewer tr,.viewer caption,.twtable th, .twtable td, .twtable tr,.twtable caption {padding:3px;}
table.listView {font-size:0.85em; margin:0.8em 1.0em;}
table.listView th, table.listView td, table.listView tr {padding:0px 3px 0px 3px;}

.viewer pre {padding:0.5em; margin-left:0.5em; font-size:1.2em; line-height:1.4em; overflow:auto;}
.viewer code {font-size:1.2em; line-height:1.4em;}

.editor {font-size:1.1em;}
.editor input, .editor textarea {display:block; width:100%; font:inherit;}
.editorFooter {padding:0.25em 0; font-size:.9em;}
.editorFooter .button {padding-top:0px; padding-bottom:0px;}

.fieldsetFix {border:0; padding:0; margin:1px 0px;}

.sparkline {line-height:1em;}
.sparktick {outline:0;}

.zoomer {font-size:1.1em; position:absolute; overflow:hidden;}
.zoomer div {padding:1em;}

* html #backstage {width:99%;}
* html #backstageArea {width:99%;}
#backstageArea {display:none; position:relative; overflow: hidden; z-index:150; padding:0.3em 0.5em;}
#backstageToolbar {position:relative;}
#backstageArea a {font-weight:bold; margin-left:0.5em; padding:0.3em 0.5em;}
#backstageButton {display:none; position:absolute; z-index:175; top:0; right:0;}
#backstageButton a {padding:0.1em 0.4em; margin:0.1em;}
#backstage {position:relative; width:100%; z-index:50;}
#backstagePanel {display:none; z-index:100; position:absolute; width:90%; margin-left:3em; padding:1em;}
.backstagePanelFooter {padding-top:0.2em; float:right;}
.backstagePanelFooter a {padding:0.2em 0.4em;}
#backstageCloak {display:none; z-index:20; position:absolute; width:100%; height:100px;}

.whenBackstage {display:none;}
.backstageVisible .whenBackstage {display:block;}
/*}}}*/
/***
StyleSheet for use when a translation requires any css style changes.
This StyleSheet can be used directly by languages such as Chinese, Japanese and Korean which need larger font sizes.
***/
/*{{{*/
body {font-size:0.8em;}
#sidebarOptions {font-size:1.05em;}
#sidebarOptions a {font-style:normal;}
#sidebarOptions .sliderPanel {font-size:0.95em;}
.subtitle {font-size:0.8em;}
.viewer table.listView {font-size:0.95em;}
/*}}}*/
/*{{{*/
@media print {
#mainMenu, #sidebar, #messageArea, .toolbar, #backstageButton, #backstageArea {display: none !important;}
#displayArea {margin: 1em 1em 0em;}
noscript {display:none;} /* Fixes a feature in Firefox 1.5.0.2 where print preview displays the noscript content */
}
/*}}}*/
<!--{{{-->
<div class='header' macro='gradient vert [[ColorPalette::PrimaryLight]] [[ColorPalette::PrimaryMid]]'>
<div class='headerShadow'>
<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
</div>
<div class='headerForeground'>
<span class='siteTitle' refresh='content' tiddler='SiteTitle'></span>&nbsp;
<span class='siteSubtitle' refresh='content' tiddler='SiteSubtitle'></span>
</div>
</div>
<div id='mainMenu' refresh='content' tiddler='MainMenu'></div>
<div id='sidebar'>
<div id='sidebarOptions' refresh='content' tiddler='SideBarOptions'></div>
<div id='sidebarTabs' refresh='content' force='true' tiddler='SideBarTabs'></div>
</div>
<div id='displayArea'>
<div id='messageArea'></div>
<div id='tiddlerDisplay'></div>
</div>
<!--}}}-->
<!--{{{-->
<div class='toolbar' macro='toolbar [[ToolbarCommands::ViewToolbar]]'></div>
<div class='title' macro='view title'></div>
<div class='subtitle'><span macro='view modifier link'></span>, <span macro='view modified date'></span> (<span macro='message views.wikified.createdPrompt'></span> <span macro='view created date'></span>)</div>
<div class='tagging' macro='tagging'></div>
<div class='tagged' macro='tags'></div>
<div class='viewer' macro='view text wikified'></div>
<div class='tagClear'></div>
<!--}}}-->
<!--{{{-->
<div class='toolbar' macro='toolbar [[ToolbarCommands::EditToolbar]]'></div>
<div class='title' macro='view title'></div>
<div class='editor' macro='edit title'></div>
<div macro='annotations'></div>
<div class='editor' macro='edit text'></div>
<div class='editor' macro='edit tags'></div><div class='editorFooter'><span macro='message views.editor.tagPrompt'></span><span macro='tagChooser excludeLists'></span></div>
<!--}}}-->
To get started with this blank [[TiddlyWiki]], you'll need to modify the following tiddlers:
* [[SiteTitle]] & [[SiteSubtitle]]: The title and subtitle of the site, as shown above (after saving, they will also appear in the browser title bar)
* [[MainMenu]]: The menu (usually on the left)
* [[DefaultTiddlers]]: Contains the names of the tiddlers that you want to appear when the TiddlyWiki is opened
You'll also need to enter your username for signing your edits: <<option txtUserName>>
These [[InterfaceOptions]] for customising [[TiddlyWiki]] are saved in your browser

Your username for signing your edits. Write it as a [[WikiWord]] (eg [[JoeBloggs]])

<<option txtUserName>>
<<option chkSaveBackups>> [[SaveBackups]]
<<option chkAutoSave>> [[AutoSave]]
<<option chkRegExpSearch>> [[RegExpSearch]]
<<option chkCaseSensitiveSearch>> [[CaseSensitiveSearch]]
<<option chkAnimate>> [[EnableAnimations]]

----
Also see [[AdvancedOptions]]
<<importTiddlers>>
[[Module Overview]]
!!!Comments
The sharp character causes the interpreter to ignore everything up to the next new line.  Making it a comment indicator.
!!!Tab completion
In interactive mode, a TAB character will cause the shell to attempt to complete the token as far as it can be uniquely determined.  If it complete the token, it will beep and a second TAB will list the possible alternatives.
!!!History
*The up and down arrows allow you to navigate through your history of commands
*History displays the last 15 commands 
*{{{!<prefix>}}} picks command matching prefix from history
*{{{!<n>}}} picks nth command
*{{{!-n}}} picks current-nth command
*{{{!!}}} picks previous command
*{{{^<s1>^<s2>^}}} executes previous command substituting s2 for s1
*{{{fc vi <n>}}} edits the nth command with vi
!!!Tokens and quoting

!!!!Tokens
The command is divided into tokens  with a specified set of meta-characters {{{SPACE, TAB, NEWLINE, ;, (, ), <, >, |, and &}}}.  Although TAB has a special interpretation when we run interactively.

!!!!Quoting and escaping
!!!!!Single quoting
{{{echo One  Two    Three      Four}}}
vs
{{{echo 'One Two    Three      Four'}}}
{{{echo 'One <Two Three Four'}}}
vs {{{echo One <Two Three Four}}}
{{{echo 'One Two    $Three    Four'}}}
!!!!Double Quoting and Variables
{{{echo "One Two    $Three    Four"}}}
before and after {{{Three=3333}}}
!!!!Escaping
{{{echo One Two \$Three Four}}}

!!![[[Shell Expansion]]]

!!!Commands and arguments
Commands are just words/tokens followed by parameters and separated by either newlines or separators.  Semicolons are the sequential separator, each command is executed in order regardless of success or failure:
{{{pwd ; funny ; who}}}

There are several styles of arguments; frequently commands support short and long forms:
short arguments have the form {{{-<character> <value>}}}
long argument have the form {{{--<key>=<value>}}}
Short form arguments can typically be combined: {{{cmd -a -b}}} is the same as {{{cmd -ab}}}

Some commands (like {{{tar}}}) also except arguments for commands without leading hyphens.  When in doubt check man page.


!!I/O Redirection
There are three ''standard i/o'' file used by Linux: standard output (stdout), standard input (stdin) and error output (errout).  Many if not most Linux commands read their input from stdin, write their output to stdout and send error messages to errout.  Bash permits redirecting these i/o ports with the 'redirection' notation:
{{{
$ cat >blue.txt
This is some txt.
$ cat blue.txt 
This is some txt.
$ cat <blue.txt 
This is some txt.
$ cat green.txt 2>error.txt  # 2> is redirection of errout
$ cat error.txt 
cat: green.txt: No such file or directory
$ cat <green.txt >lime.txt 2>error.txt
bash: green.txt: No such file or directory  #note error message from bash not cat.
$ echo "Steve Hirst - Error log" >>error.txt # appends to file
$ echo "Blue Text" >blue.txt # overwrites file
$ cat >blue.txt <<EOF
> This is some kind of data
> This is a second row
> EOF

}}}


!!!Completion and exit status
Commands all return a completion code between 0-255; these are used by Bash to determine if the command "succeeded" or not.  0 indicates success all other codes indicate failure.  The variable substitution {{{$?}}} evalutates to the return code of the last command.

Return codes 128-255 are generally reserved for commands that were halted by system signals;  commands that end with error codes should use the range 0-125; Bash uses return code of 127 to indicate command not found; and 126 if the file does not have execute permissions.
{{{
$ cat green.txt
cat: green.txt: No such file or directory
$ echo $?
1
$ cat blue.txt 
This is some txt.
$ echo $?
0

}}}
!!!Combining commands
We can combine commands into a single line (or multiple lines if we terminate the line with a {{{\}}}) in one of four ways.
!!!!sequencing
A semicolon {{{;}}} indicates that commands are to be executed sequentially
{{{
$ cat a.txt ; echo "Extra line" >>a.txt ; cat a.txt
A file
A file
Extra line
}}}
!!!!piping
A ''pipe'' ({{{|}}} connects the std output of the first command to the std input of the second command:
{{{
cat <<EOF | cat
> One
> Two
> EOF
One
Two

}}}
!!!!or-execution
A or combining executes the second only if the first fails.  It halts with the first command that succeeds. It returns the return code of the last command executed.  So it returns 0 unless all the commands fail.
{{{
$ cat xxx.txt || cat blue.txt || cat zzz.txt
cat: xxx.txt: No such file or directory
This is some kind of data
This is a second row

$ cat xxx.txt || cat zzz.txt || cat blue.txt
cat: xxx.txt: No such file or directory
cat: zzz.txt: No such file or directory
This is some kind of data
This is a second row
$ echo $?
0
cat xxx.txt || cat zzz.txt 
cat: xxx.txt: No such file or directory
cat: zzz.txt: No such file or directory
$ echo $?
1
}}}
!!!!and- execution
The and-combiner ({{{&&}}}) executes command until one files.  The return code will be the return code of the command that fails or 0.
{{{
$ cat xxx.txt && cat blue.txt
cat: xxx.txt: No such file or directory
$ echo $?
1
$ cat blue.txt && cat blue.txt && cat xxx.txt && cat blue.txt 
This is some kind of data
This is a second row
This is some kind of data
This is a second row
cat: xxx.txt: No such file or directory
$ echo $?
1

}}}
!!!!useful environment variables
| SHELL | path to command shell |
| USER | id of logged in user |
| PATH | search path for commands |
| PWD | current working directory |
| HOME | the home directory of current user ({{{~}}}) |
| OLDPWD | prior working director |
And the special values such as {{{$?}}} and {{{$$}}} (return code and processid)


!!!Finding commands and finding out about commands
!!!!man, info and help also Google
example {{{date --help}}} vs {{{man date}}} vs {{{info date}}}
 Note: {{{q}}} exits, for selecting menu moves about and {{{P}}} returns you to previous location.

The {{{man}}} command supports apropos feature to find possible commands.
{{{man -k sort}}}


!!!Command location and the PATH variable
The shell takes the first token in a command as the command name.  If the first token is a file path to a file, it will execute that command.
Otherwise, it will: check if it is an internal command and if so execute
Finally, it will search the directories in the $PATH variable in order looking for the command.

{{{--version}}} switch, {{{which}}} and {{{type}}} commands

Linux is an operating system like Windows, Solaris, z/OS, etc; it is the ''layer'' of software which supports all other software systems, and it provides the unifying interface for devices and users.

Linux is different from other systems in that is has more varieties because the various components can selected from a variety of sources.  Partly this is an architectural choice on the part the designers of the ''Linux kernel''; partly it is a result of the //open// nature of the Linux development process.

An instance if the Linux OS is composed of:
* a kernel
* a set of device drivers
* library providing an api for programs
* one or more command shells
* a collection of utility commands
* a set of configuration choices
* optionally a implementation of x-windows, and one or more window managers and desktop managers.

While it is possible to build a Linux system from scratch starting with source code and a compiler, most of the time Linux systems are built from a specific ''distribution.'' Some of these distributions (such as __Red Hat__) are commercial and supported by large organizations, some (__Ubuntu__) are free and supported by not-for-profit organizations, others are smaller and supported by individuals or small organizations.  

Distributions typically bundle together all of the various components and provide some sort of installation/configuration support.

The Linux Kernel is developed by the Linux Foundation (http://www.linuxfoundation.org) and was begun under the leadership of Linus Torvalds in 1991.  The Linux Kernal was developed to be an independent and open implementation of a Unix Kernel.  The kernel provides the primary framework for abstracting away the specifics of the individual computer's hardware.

In the first distribution of the kernel, Torvalds wrote:
>Sadly, a kernel by itself gets you nowhere. To get a working system you need a shell, compilers, a library etc. These are separate parts and may be under a stricter (or even looser) copyright. Most of the tools used with linux are GNU software and are under the GNU copyleft. These tools aren't in the distribution - ask me (or GNU) for more info.

The library and utilities are primarily developed and maintained by the GNU foundation (http://www.gnu.org/) which is an older project (1984) begun by Richard Stallman to create open versions of standard Unix commands and tools.

Other components are developed and maintained by specialized organizations, distribution providers and individuals.
[[GettingStarted]]

[[Module Overview]]
!!! Session 1: Becoming a Linux user
# what is [[Linux]]
# using the [[command shell]]
# [[user login, permissions and security]]
# [[process execution]], halting, and monitoring, signals, crontab
# [[file system]] basics;  directory and file naming standard,
!!! Session 2: Working on production systems with Linux
# [[working remotely]] with multiple servers-ssh and file transfers
# useful [[file manipulation]] commands and command piping
# [[regular expressions]], glob, grep, fgrep, sed
!!! Session 3: Creating and maintain shell scripts
# [[editing]]  ed, vim
# [[bash scripts]] - with standards  as well as live examples



working on the Pearson integration servers
Introduction to Linux
After the command line is broken into tokens, it goes through a sequence of expansion technologies before it executes the command.

!!!!Brace expansion
{{{$ ls *.{txt,sh} }}}
{{{$ echo {a..m} }}}
{{{$ echo {A..Z}{0..9} }}}
{{{$ echo temp{0..3}.{idx,dat} }}}
{{{$ {cp text.txt{,.bak} }}}
{{{
$ ls local/{osa,sps}/data/va/ssa/sprg09

local/osa/data/va/ssa/sprg09:
four  osa_student_extract_srch.txt

local/sps/data/va/ssa/sprg09:
four
}}}
A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged.
{{{
$ echo {one,two,three}
one two three
$ echo {one,two three}
{one,two three}
$ echo {one,"two three"}
one two three
}}}
!!!!Tilde expansion
|{{{~}}}|Home directory of the logged in user.|
|{{{~name}}}|Home directory of named user, ie. /home/name.|
|{{{~+}}}|Current working directory ($PWD).|
|{{{~-}}}|Previous working directory ($OLDPWD). |

If the expansion doesn't work; i.e if the {{{name}}} is not the name of an defined user or there is no previous working directory, the string is left untouched.
{{{
$ echo ~steve ~root ~albert

/home/steve /root ~albert
}}}
Note tilde expansion is done after brace expansion:
{{{
$ echo ~{steve,root,albert}

/home/steve /root ~albert
}}}
!!!!File name expansion
Bash scans each token for the characters ‘*’, ‘?’, and ‘[’. If one of these characters appears, then the token is regarded as a pattern, and replaced with an alphabetically sorted list of file names matching the pattern. If no matching file names are found, the word is left unchanged.

An ''?'' matches any single character:
{{{
$ echo a?.txt
aa.txt a..txt az.txt
}}}
A "*" matches 0 or more instances of any character; but the zero pattern won't typically match files begining with a ''.''
{{{
$ echo *.txt
aa.txt a.txt a..txt az.txt bash-notes.txt err.txt Greek Words.txt temp.txt text.txt t.txt
$ ls .txt 
.txt
}}}

The {{{ [<range>] }}} lets use specify a range of characters which can be matched:
{{{
$ echo a?.txt
aa.txt a..txt az.txt aZ.txt

$ echo a[a-z].txt
aa.txt az.txt

$ echo a[AZ].txt
aZ.txt

$ echo a[aA].txt
aa.txt
}}}
The {{{ [!range] }}} matches characters not in the range.
{{{
$ echo a[!a-z].txt
a..txt aZ.txt
}}}
!!!!Variable expansion
The base environment supports variables.  The {{{$}}} signals that the variable is to be evaluated.
The assignment statement has the form {{{var=value}}}. We can remove the assignment (and remove the variable by using the ''unset'' command; but, we can't use ''set'' to assign values.
{{{
$ echo $myVar

$ myVar="First Value"
$ echo $myVar
First Value
$ unset myVar
$ echo $myVar


}}}
{{{
echo $PWD
/home/steve/lnfsm
echo ${PWD}
/home/steve/lnfsm
echo $PWD local
/home/steve/lnfsm local
echo $PWD/local
/home/steve/lnfsm/local
\echo $PWDlocal

steve@Twisty:~/lnfsm$ echo ${PWD}local
/home/steve/lnfsmlocal

}}}

{{{
steve@Twisty:~/lnfsm$ echo ${myVar:-default}
default
steve@Twisty:~/lnfsm$ myVar=non-default
steve@Twisty:~/lnfsm$ echo ${myVar:-default}
non-default
steve@Twisty:~/lnfsm$ unset myVar
steve@Twisty:~/lnfsm$ echo ${myVar:-default}
default

}}}
{{{
steve@Twisty:~/lnfsm$ echo $PWD
/home/steve/lnfsm
steve@Twisty:~/lnfsm$ echo ${PWD}
/home/steve/lnfsm
steve@Twisty:~/lnfsm$ 3echo ${#PWD}
bash: 3echo: command not found
steve@Twisty:~/lnfsm$ echo ${#PWD}
17
steve@Twisty:~/lnfsm$ echo ${#PWD}
17
steve@Twisty:~/lnfsm$ echo ${PWD#~}
/lnfsm
steve@Twisty:~/lnfsm$ echo ${PWD#/h*/}
steve/lnfsm
steve@Twisty:~/lnfsm$ echo ${PWD##/h*/}
lnfsm

$ test="Rain in Spain"
steve@Twisty:~$ echo ${test/ain/ixn/}
Rixn/ in Spain
steve@Twisty:~$ echo ${test/ain/ixn}
Rixn in Spain
steve@Twisty:~$ echo ${test//ain/ixn}
Rixn in Spixn

}}}

{{{
steve@Twisty:~/lnfsm$ myVar="Now is the time for all good men to come to the aid of their country."
steve@Twisty:~/lnfsm$ echo ${myVar#*t}
he time for all good men to come to the aid of their country.
steve@Twisty:~/lnfsm$ echo ${myVar##*t}
ry.
steve@Twisty:~/lnfsm$ echo ${myVar%t*}
Now is the time for all good men to come to the aid of their coun
steve@Twisty:~/lnfsm$ echo ${myVar%%t*}
Now is
steve@Twisty:~/lnfsm$ echo ${myVar:11:17}
time for all good

}}}

!!!!Execution expansion

{{{
$ cat aa.txt 
A file
$ echo One $(cat aa.txt) Three
One A file Three
$ echo One `cat aa.txt` Three
One A file Three
$ echo One `cat aaa.txt` Three
cat: aaa.txt: No such file or directory
One Three
$ 
}}}
The Advanced guide has a good section (29) on debugging strategies.

the bash command has two options that let you run scripts in visible mode {{{v}}} and {{{x}}}
{{{bash -v scriptname}}} echoes each command before executing it. 

{{{bash -x scriptname}}} echoes the result each command in an abbreviated manner.

{{{
$ bash -xv myScript.bsh green.txt grey.txt
cat $1 $2 >$tempDir/comb.txt
+ cat green.txt grey.txt
myScript.bsh: line 1: /comb.txt: Permission denied
wc $tempDir/comb.txt
+ wc /comb.txt
wc: /comb.txt: No such file or directory
./problem.bsh
+ ./problem.bsh
myScript.bsh: line 3: ./problem.bsh: No such file or directory
}}}

{{{
$ bash -xv wrapper.bsh myScript.bsh green.txt red.txt
#!/bin/bash
#  Set up temp dir ; redirector error log
tempDir=temps-`date +%F`-$$
date +%F
++ date +%F
+ tempDir=temps-2009-12-15-1275
mkdir $tempDir
+ mkdir temps-2009-12-15-1275
exec 2>>error-`date +%F`-$$.log
+ exec
date +%F
++ date +%F
 5  7 37 temps-2009-12-15-1275/comb.txt
}}}

{{{
$ more error-2009-12-15-991.log
script=$1
+ script=myScript.bsh
shift
+ shift
. $script
+ . myScript.bsh
cat $1 $2 >$tempDir/comb.txt
++ cat green.txt grey.txt
cat: grey.txt: No such file or directory
wc $tempDir/comb.txt
++ wc temps-2009-12-15-991/comb.txt
./problem.bsh
++ ./problem.bsh
myScript.bsh: line 3: ./problem.bsh: No such file or directory

# clean up
rm -rf $tempDir
+ rm -rf temps-2009-12-15-991
}}}

Inserting set -u  in a script gives an unbound variable error message at each attempt to use an undeclared variable.  This can either be a debugging step or left in as a standard within the code:
{{{
nl uninit-var.bsh 
     1	#!/bin/bash
     2	echo Here is $MyVar
     3	set -u
     4	echo Here is $MyVar
}}}
When executes:
{{{
$ ./uninit-var.bsh 
Here is
./uninit-var.bsh: line 4: MyVar: unbound variable
}}}
!!!!Creating scripts
Making a hello world script:
{{{
$ cat >hello.bsh
#!/bin/bash
echo "Hello, world."
^d
$ chmod a+x hello.bsh 
$ stat -c %A hello.bsh 
-rwxr-xr-x
$ ./hello.bsh 
Hello, world.
$ cat hello.bsh 
#!/bin/bash
echo "Hello, world."
}}}

First step create file containing two lines: (1) {{{#!/bin/bash}}} and (2) a bash command {{{echo "Hello, world."}}}.
The command {{{chmod a+x hello.bsh}}}; then typeing the path to the script {{{./hello.bsh}}} cause the command to execute.

Notes:
1. the first line {{{#!/bin/bash}}} is more than a comment even though it starts with {{{#}}}.  The text must be a full bath to the bash executable.  Changing {{{bash}}} to {{{bush}}} will cause the command to fail to execute.
{{{
$ ed hello.bsh 
33
1
#!/bin/bash
s/bash/bush/
p
#!/bin/bush
w
33
q
$ ./hello.bsh 
bash: ./hello.bsh: /bin/bush: bad interpreter: No such file or directory
}}}
2. If you type the command instead of a path, bash will search for the command in the PATH search path and fail to find it.
{{{
$ ./hello.bsh 
Hello, world.
$ hello.bash
bash: hello.bash: command not found
$ hello.bsh
bash: hello.bsh: command not found
$ ~/lnfsm/hello.bsh 
Hello, world.
}}}

A script can be created just by putting command lines sequentially in the file.  A script created this way can be quite powerful.  An example is the following script with calcultes z-batch numbers:
{{{
#!/bin/bash
tr ' ' '_' <summary.dat >de-blanked.dat
cut -c 1-29  <de-blanked.dat >group-key.dat
cut -c 30-60 <de-blanked.dat >data.dat
paste group-key.dat data.dat >delimed.dat
sort <delimed.dat >sorted.dat
uniq -c -w 30 <sorted.dat >counted.dat
./calc-zbatch.bsh <counted.dat >batchno.dat
sort group-key.dat | uniq >keys.dat
paste keys.dat batchno.dat >assigned.dat
join -j 1 sorted.dat assigned.dat | tr -d ' ' | tr '_' ' ' >batched.dat
}}}

Putting a command in a script file it will operate the same when the script with two main exceptions:
# alias definitions do not carry forward.
# local variable bindings do not carry forward:
!!!Environment and local variables
The semantics of variables is the same in bash scripts as in command line but they do not share ''local variables''.
Consider:
{{{
#!/bin/bash
echo "Hello, $World."
}}}
Defining a the variable {{{World}}} does not carry into the script's environment; unless, we {{{export}}} into the global environment:
{{{
$ export World
$ ./hello.bsh 
Hello, Steve's world.
}}}
Conversely setting variables in a bash script does not  impact the calling environment:
{{{
$ unset World
$ cat hello.bsh 
#!/bin/bash
World="script world"
echo "Hello, $World."
$ echo $World

$ ./hello.bsh 
Hello, script world.
$ echo $World

}}}
This is true even if you include an export command in the script;  the binding goes away when the script completes:
{{{
$ cat hello.bsh 
#!/bin/bash
export World="script world"
echo "Hello, $World."
$ echo $World

$ ./hello.bsh 
Hello, script world.
$ echo $World

}}}
If you want a scripts variable bindings to impact an environment you must ''source'' it rather than calling it.  The command source can also be typed as a '.'  
{{{
$ cat hello.bsh 
#!/bin/bash
World="script world"
export World
echo "Hello, $World."

$ unset World
$ echo $World

$ ./hello.bsh 
Hello, script world.
$ echo $World

$ source hello.bsh 
Hello, script world.
$ echo $World
script world
$ unset World
$ . hello.bsh 
Hello, script world.
$ echo $World
script world
}}}
Sourcing a script is like a textual inclusion, or typing them in an interactive environment.  This means that all variable bindngs are visible.
{{{
$ cat hello.bsh 
#!/bin/bash
echo "Hello, $World."
$ World="my world"
$ . hello.bsh 
Hello, my world.
}}}
!!!!Positional parameters
In addition to variables, we can bash variables to scripts through positional paramters:
These variables are numbered starting with 0, with 0 being the command, 1 being the first argument ,2 the second, etc:
{{{
$ cat hello.bsh 
#!/bin/bash
echo "Hello, $1."
$ ./hello.bsh "Steve's world"
Hello, Steve's world.
$ . hello.bsh "Steve's world"
Hello, Steve's world.
$ . hello.bsh "Jack's world"
Hello, Jack's world.
}}}
Notice, parameters are passed both when calling and when sourcing a script.
Also, when you have more than 9 arguments you need to put the position in braces; remember you can always use brace notation:
{{{
$ cat position.bsh 
#!/bin/bash
echo $10-${10}-${1}0
$ ./position.bsh One Two Three Four Five Six Seven Eight Nine Ten Eleven Twelve
One0-Ten-One0
}}}
In sourcing, if you don't specify an agument list, the current set of positional parameters is available to the script.

Functions can be defined in bash scripts and they use the same positional notation for their arguments.  Note they hide the calling positional paramaters for the duration of the call:
{{{
$ cat sample-function.bsh 
#!/bin/bash
reverse() {
echo $2\\$1
}
reverse a b
$ ./sample-function.bsh 
b\a
$ reverse 4 5
bash: reverse: command not found
$ . sample-function.bsh 
b\a
$ reverse first second
second\first

}}}
Notice that function definitions are treated like variable bindings in that in a called script, the definition goes away after the call.  In the sourced execution, the definition enters the current execution environment.

There are two special variable notations that refer to the complete set of positional arguments $* and $@
{{{
$ cat position.bsh 
#!/bin/bash
first(){
  echo $1
}
echo $10-${10}-${1}0
echo -n 'All aruguments as a single token $*: '
first "$*"
echo -n 'All arguments as separate tokens $@: '
first  "$@"
$ ./position.bsh One Two Three Four Five Six Seven Eight Nine Ten Eleven Twelve
One0-Ten-One0
All aruguments as a single token $*: One Two Three Four Five Six Seven Eight Nine Ten Eleven Twelve
All arguments as separate tokens $@: One
$ . position.bsh One Two Three Four Five Six Seven Eight Nine Ten Eleven Twelve
One0-Ten-One0
All aruguments as a single token $*: One Two Three Four Five Six Seven Eight Nine Ten Eleven Twelve
All arguments as separate tokens $@: One
}}}
Note: failing to quote the paramters (as in {{{first "$@"}}}) causes bash to re-parse the line into tokens and defeats the purpose.

The {{{shift}}} command moves the numbered arguments to the left (decrement)  as in:
{{{
#!/bin/bash
# left shift args
echo "Args before shift: $*"
echo "Number of args: $#"
shift
echo "Args after shift: $*"
echo "Number of args: $#"
}}}

{{{
$ ./shift-demo.bsh One Two Three Four
Args before shift: One Two Three Four
Number of args: 4
Args after shift: Two Three Four
Number of args: 3
}}}


!!!! The {{{exec}}} command
The exec command has two functions:
* exec with script as an argument terminates the current scripts and then sources the target.  The current script is never returned to.
* exec with redirections, changes the redirection of stdin, stdout or stderr for the following script.
!!!! Sample application
The wrapper function adds temp-directory management and error-logging to any script.
{{{
#!/bin/bash
#  Set up temp dir ; redirector error log
tempDir=temps-`date +%F`-$$
mkdir $tempDir
exec 2>>error-`date +%F`-$$.log
script=$1
shift
. $script
# clean up
rm -rf $tempDir
}}}

{{{
$ ./wrapper.bsh shift-demo.bsh One Two Three Four
Args before shift: One Two Three Four
Number of args: 4
Args after shift: Two Three Four
Number of args: 3
}}}

{{{
$ ./wrapper.bsh myScript.bsh One
0 0 0 temps-2009-12-15-624/comb.txt
steve@Steamy:~/lnfsm$ ls -lh  error-2009-12-15-*
}}}
{{{
$ more myScript.bsh
cat $1 $2 >$tempDir/comb.txt
wc $tempDir/comb.txt
./problem.bsh
}}}
!!!! ''exit codes''
The builtin {{{exit}}} takes a return code as an argument and halts the script returning the argument as the results code.

!!!! Subshell
Surrounding a command list in parenthesis creates a subshell; this is quite useful when you want to temporarily change directories.  Then at the end of the commands the current shell resumes with its current working directory.  However, local variables are not shared with the subshell.
{{{
 pwd; ( cd ../Dropbox/Linux\ Class/Linux\ scripts/;echo "Directory: `pwd`"; ls ) ; pwd
}}}
!!! Tests
The test operators in bash are typically of the the form {{{-<test> <arg>}}} or {{{<arg1> -<test> <arg2>}}} and they return exit codes 0 for success, non-zero for failure.  You can invert by preceeding it with an exclamation point {{{!}}}.  In general, you will enclose comparisons in {{{[[}}} and {{{]]}}} ( they can be done with out enclosing them or (particularly in older scripts) with single brackets {{{[}}} and {{{]}}}.  Double parenthesis are used for numeric tests.  Note, when testing pair strings only null strings are false.
There a several file tests (See Advanced Bash-Scripting Guide 7.2} including
* {{{-e file}}}   -- file exists
* {{{-d file }}}  -- file is a directory 
* {{{-s file }}}  -- file is not empty (size>0)
* {{{-w file }}} -- script can write to file

You can compare numbers:  (Section 7.3) Other scripts
* {{{ a -gt b }}} -- greater than
* {{{ a -lt b }}} -- less than
* {{{ a -eq b }}} -- equal
And Strings
* {{{ a == b }}} -- equal or  when enclosed in double brackets b can be a regex.
* {{{ a != b }}} -- not equal or when enclosed in [[ ]] a doesn't match b.

You can do numeric comparisons  expression evaluation within double parenthesis see section 9.7
* {{{(( a < b }}}} -- a less than b numerically ; ''a'' and ''b'' can be simple arithemetic expressions +,-,*,/ and single parens
* {{{((( a >= b)) }}}
* {{{ (( a )) }}}  returns value of a

Within the double parens tests can be combined with ''and'' {{{ && }}} and ''or'' {{{ || }}} operators.

The test construct can be used with the command list operators to implement conditional logic:
{{{
$ [[ -e numbers2.txt || -e second-new.txt ]] && echo "One of the files exist"
One of the files exist

$ [[ -e numbers.txt && -e second-new.txt ]] && echo "Both files exist"
Both files exist
}}}


!!!! [[control structures]]

!!!! [[readline programming]]

!!!! Debugging
[[bash debugging]]
!!!Bourne Again Shell
The primary way you will communicate with a Linux system is via a command interpreter which is generally referred as a command shell or simple shell.  The most frequently used shell, is the //bash shell// or the Bourne again shell. 

The main reason you want to interact with a Linux is to execute GNU/Linux commands to manipulate files, run jobs etc.  An command shell is a program which will interpret its input as commands and invoke those commands.  //bash// is a rewrite Steve Bourne's original default shell for Unix under the GNU license. 

Things to remember about bash:
* there is nothing special or privileged about bash -- it is a c-program that reads commands and executes them.
* any linux program can be executed from the shell, most usefully if it follows standard i/o conventions, argument processing etc.
* a shell can be run interactively where it accepts input from keyboard and displays results, or it can take it's input from a file (script execution).

!!!Shell customization
When you login into a Linux system locally or remotely, your account is associated with a command shell.  At startup, the shell will execute one or more scripts which initialize //aliases// and //environment variables//.  Then you will be greeted with a "shell prompt" and it will wait for you to enter a command.  

Bash reads commands from it's input and runs commands regardless of the source of the input (file or interactive).  However, when a bash interpreter starts up it is one of four possible states:
# an interactive login shell; when the shell is started by your logging into Linux
# an interactive non-login shell when a second (or third or ...) shell is started after logging in.
# a batch non-login shell: when you execute a bash script
# a batch login shell (a special and rare case); when you specifiy the {{{--login}}} option in starting bash

 Bashrc is not to be confused with bash_profile— the difference is that bashrc runs every time you "become" that user by any means (including su), whereas bash_profile does not. bash_profile is run only when the user logs in proper.

Regardless of which path, the //run commands// customize the session for your environment and preferences.  The customizations come in the form of:
# shell options, set through the {{{shopt}}} command
# aliases for commands
# environment variables

Aliases are command shortcuts defined for your convience; frequently they are just used to set default options.  For example, the {{{alias ls='ls --color=auto'}}} makes the color display of file listings standard rather than optional.  Alias ''do not'' impact the execution of commands because they are not passed to //sub-shells.//

Environment variables are made available gobally and can alter both the execution of the shell itself and other commands.  The command {{{printenv}}} displays the global environment variables; one of the most significant environment variables is the {{{PATH}}} variable which specifies the list of directories which are searched for command executables:
{{{
PATH=/home/steve/bin:/opt/emperor/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
}}}
Std .bash_profile
{{{
# Get the aliases and functions
if [ -f ~/.bashrc ]; then
	. ~/.bashrc
fi

# User specific environment and startup programs

USS_LOC=/hosting/uss
NFSDIR=/lnfsm
ORACLE_HOME=/home/oracle/app/oracle/product/10.2.0/client_1
PATH=$PATH:$ORACLE_HOME/bin:$HOME/bin
export USS_LOC NFSDIR ORACLE_HOME PATH

unset USERNAME
}}}

Optional .bashrc
{{{
# User specific aliases and functions

alias cdems='cd /hosting/uss/ems/scripts/va/ssa/fall06/'
alias cdepen='cd /hosting/uss/epen/scripts/va/ssa/fall06/'
alias cdint='cd /hosting/uss/int/scripts/va/ssa/fall06/'
alias cdosa='cd /hosting/uss/osa/scripts/va/ssa/fall06/'
alias cdpde='cd /hosting/uss/pde/scripts/va/ssa/fall06/'
alias cdrs='cd /hosting/uss/rs/scripts/va/ssa/fall06/'
alias cdsorm='cd /hosting/uss/sorm/scripts/va/ssa/fall06/'
alias cdsps='cd /hosting/uss/sps/scripts/va/ssa/fall06/'

# perforce variables
export PATH=$PATH:$HOME/perforce
export P4PORT=perforce.ic.ncs.com:1424
export P4CLIENT=ussdev_promote
export P4USER=build

# Java Env Vars
export CLASSPATH='/hosting/uss/int/java/lib/va/ssa/fall07/'

# Source global definitions
if [ -f /etc/bashrc ]; then
	. /etc/bashrc
fi
}}}
!!!Shell Execution
The basics of shell execution is as follows:
# It reads a line from the input (file or interactive) discarding commented text
# Breaks the line  into tokens following the quoting rules and expands alias definitions
# Organizes the tokens into commands and arguments (Simple commands)
# Performs Shell Expansions
# Implements i/o redirections and removes redirection operators and arguments from argument list
# Looks up the command using the PATH global environment variable
# Optionally waits for the Exit status

Some commands: {{{cat}}}, {{{echo}}}, {{{date}}}, {{{cp}}}, {{{ls}}}
[[Interacting with the shell]]
!!!! if then
The ''if then'' structure of bash ends with ''fi'' and includes ''elsif'' and ''else'' constructions.
From the man page

>The if list is executed.  If its exit status is zero, the then list is executed.  Otherwise, each  elif  list  is executed  in  turn,  and if its exit status is zero, the corresponding then list is executed and the command completes.  Otherwise, the else list is executed, if present.  The exit status is the exit status of the  last  command executed, or zero if no condition tested true.

Where 
>A list is a sequence of one or more pipelines separated by one of the operators ;, &, &&, or ││

{{{
#!/bin/bash
DIR="$1"
 
if [[ $# -ne 1 ]] #test for number of arguments
then
	echo "Usage: $0 {dir-name}"
	exit 1
fi
 
if [[ -d "$DIR" ]] # test if argument is directory
then
	echo "$DIR directory  exists!"
        ls $DIR
else
	echo "$DIR directory not found!"
fi

}}}
!!!! for loop
The for loop in bash loops over a set of values in a list, which can include file wild carding:
{{{
for arg in [list]
do
 command(s)...
done 
}}}

Sample
{{{
#!/bin/bash
# iterate over everything that starts with err plus the arguments
for LOG_FILE in err* $@
do
  echo "Checking $LOG_FILE"
  stat -c %s $LOG_FILE
done
}}}

Running: 
{{{
$ ./check-logs.bsh blue.txt green.txt red.txt 
Checking error_log
300853
Checking error.log
76
Checking error.txt
42
Checking err.txt
73
Checking blue.txt
7
Checking green.txt
stat: cannot stat `green.txt': No such file or directory
Checking red.txt
25
}}}

Also, by using the execute function you can process parts of a file using a for loop:
{{{
#!/bin/bash
for LINE in `cut -d ' ' -f 1,3 donate.rec | sed "s/ /:/g"`
do
   echo Donation from ${LINE/:/ for }
done

}}}
Output:
{{{
$ ./for-fields.bsh 
Donation from Bay for 500000
Donation from Cruella for 725000
Donation from Harold for 1300050
Donation from Jack for 250000
Donation from More for 2300000
Donation from Greedy for 023
}}}

Using brace expansion for loop control:
{{{
#!/bin/bash
for a in {1..10}
do
  echo -n "$a "
done  
}}}
!!!! while do/until do
{{{
while list; do list; done
until list; do list; done
}}}
!!!! case
{{{
#!/bin/bash
# explain type of argument
case $1 in 
(*.txt) echo "$1 is a text file";;
(*.xml) echo "$1 is an xml file";;
(*.png|*.svg) echo "$1 is an image file";;
(*) echo "I wonder what $1 is.";;
esac
}}}
Note: the leading parenthesis on the test is optional; and you will frequently see code that leaves it off.
{{{
$ ./suffix-info.bsh aa.txt 
aa.txt is a text file
$ ./suffix-info.bsh Sketch12.svg 
Sketch12.svg is an image file
$ ./suffix-info.bsh harry.xml 
harry.xml is an xml file
$ ./suffix-info.bsh empty.html 
I wonder what empty.html is.
}}}
!!!! break
The {{{break}}} statement breaks you out of the innermost control loop
{{{
#!/bin/bash
# nested for loops; break gets exit inner loop
for a in {1..10}
do
  for b in {1..10}
  do
     echo -n "<$a,$b> "
     if [[ $a -lt $b ]] ; then break ; fi
  done
done  

echo; echo
}}}
{{{ed}}} and {{{vi}}} are convenient ways to edit files with out transferring them back and forth from your workstation.

!!!!ed
{{{ed}}} is the base/original editor for Unix; it is line-oriented, terse and gives almost no visual feedback.  But it does support regular expressions 
{{{
$ ed donate.rec 
154

}}}
Here is an example session
{{{
     1	$ ed donate.rec 
     2	154
     3	p
     4	Greedy Guts 023 Norway
     5	n
     6	6	Greedy Guts 023 Norway
     7	1,$p
     8	Bay Ching 500000 China
     9	Cruella Lumper 725000 Malaysia
    10	XXXX YYYY 2222 Paraquay
    11	Jack Arta 250000 Indonesia
    12	More Money 2300000 Norway
    13	Greedy Guts 023 Norway
    14	2p
    15	Cruella Lumper 725000 Malaysia
    16	3
    17	XXXX YYYY 2222 Paraquay
    18	s/XXXX/Harold/p
    19	Harold YYYY 2222 Paraquay
    20	s/YYYY/Earwicker/
    21	s/2222/1300050/p
    22	Harold Earwicker 1300050 Paraquay
    23	w
    24	164
    25	q
}}}

Line  1 -- start ed on the file donate.rec
Line  2 -- ed replies with the number of characters in donate.rec
Line  3 -- p the command to print the ''current line''
Line  5 -- n print the current line preceeded by a line number (ed is positioned at the last line in the file)
Line  7 -- print line 1 through the last line in the file
Line 14 -- move to line 2 and print it
Line 16 -- move to line 3 (default is to print it)
Line 18 -- substitute the pattern XXXX with the string Harold and print it. (the pattern can be a full regex)
Line 20 -- substitute the string YYYY with Earwicker
Line 21 -- substitute the string 2222 with 1300050 and print line
Line 23 -- write file
Line 24 -- ed replies with the number of characters written
Line 25 -- q

The ''diff -e'' command will output a script of ed commands that will change the first file to the second:
{{{
diff -e read*
3,4c
while  read first second third; do
   echo "I read: $first followed by $second followed by $third"
.
}}}


Note the {{{sed}}} command's substitute command ( {{{s}}} ) works much the same as the ed substitute command but is applied to the the whole file a line at a time.   See section C.1 of Advanced Bash-scripting Guide
!!!!vim (or vi improved
The {{{vim}}} or {{{vi}}} command starts the Linux /visual/ editor.  {{{vi donate.rec }}} starts it with to edit donate.rec. 
The cursor keys move you a around the text 
The command {{{i}}} put you into insert move and the /escape key/ gets you out of the mode.
The command {{{o}}} opens a new line below and puts you into insert mode.
The command {{{/}}} followed by a regex finds the first line matching the exrpression
The command {{{dd}}} deletes the line
The command {{{:w}}} writes modified file over opened file
The command {{{:w}}} writes modified file to specified file
The command {{{:q}}} quits but will complain if you haven't written the file
The command {{{:q!}}} quits without complaint

http://www.tuxfiles.org/linuxhelp/vimcheat.html


!!! awk
Awk is programming language aimed at text processing. Section C.2 of Advanced Bash-scripting guide
{{{
awk '{print}' error_log

awk '(NR >= 10 ) && (NR <= 20) { print } ' error_log
}}}

using awk with a command processes the file a line a time.  The {{{print}}} command is the the same as {{{print $0}}} prints the current line.
NR is a builtin {{{awk}}} variable that is the line number of the  file.
{{{ls *.* | sed "s/^.*\.//" | sort | uniq}}}
!!!Lines
* sort
Note many of the current version of sorts in Linux is sensitive to the ''locale'' settings.  In particular, the environment LC_ALL, LANG and LANGUAGE.  If you get help on {{{sort}}} the concluding statement is:
{{{
*** WARNING ***
The locale specified by the environment affects sort order.
Set LC_ALL=C to get the traditional sort order that uses
native byte values.
Report bugs to <bug-coreutils@gnu.org>.
}}}

{{{sort file}}} places file in order based on character values  {{{sort-me.txt}}}
{{{sort -b }}} ignores leading blanks
{{{sort -d }}} does dictionary sort
{{{sort -f }}} folds lowercase into uppercase (case insensitive sort)
{{{sort -r }}} puts in reverse order
{{{sort -n }}} sort numerically {{{numbers.txt}}}

{{{sort -k v}}} sorts blank separated file by field position {{{donate.rec}}}
{{{sort -k 3 donate.rec}}} v. {{{sort -n -k 3 donate.rec}}}

{{{sort -k 1.2 sort-me.txt }}}

* uniq
Remove duplicate lines
{{{
sort  sort-me.txt | uniq 
sort -f sort-me.txt | uniq 
sort -f sort-me.txt | uniq -i
}}}

{{{shuf sort-me.txt}}}

* fold
fold divides lines -w width if -s divide at space
{{{fold -w 20 CosmicComputer.txt | less }}}
{{{fold -sw 20 CosmicComputer.txt | less }}}

Note width is in columns (takes multi-byte characters into account) -b says just use byte count.

* nl
number lines:
{{{nl sort-me.txt }}}
{{{nl -nrz sort-me.txt }}}
{{{nl sort-me.txt | sort -bfk 2}}}


!!!Characters
* hexdump
* tr (shift case, remove \r)
{{{ tr 'a-z' 'A-Z' <CosmicComputer.txt | less }}}
{{{tr '.,?;:!' '*' <CosmicComputer.txt | less}}}
{{{tr -d '0-9' <donate.rec }}}
{{{tr -d '\r' <dos.txt | less -u}}}
* expand and unexpand
{{{expand -t 4 tabs.txt}}}
{{{unexpand -t 6 no-tabs.txt}}}
!!!Fields
* cut, paste 
{{{cut -c 4-6 sort-me.txt }}}
{{{cut -d ' ' -f 1,3 donate.rec }}}
{{{cut -d ',' -f 2,4,5 pmd-report.csv}}}
{{{paste red.txt blue.txt}}}

* join
{{{
$ sort -k 4 donate.rec >donate.sorted
steve@Twisty:~/lnfsm$ join -a 1 -1 4 -2 2 donate.sorted country.code 
}}}
Note both files must be sorted on join-key
* colrm 
Removes columns between two character postions
{{{ colrm 10 50 <CosmicComputer.txt | less }}}
* column
format for multi-column format
{{{column donate.rec}}}
!!!Information
* find
{{{ find . -iname "err*"}}}
{{{find . -iname "*.msg"}}}
{{{find . -iname "*.txt"}}}
{{{find local -iname "*.txt"}}}
* wc
{{{
wc CosmicComputer.txt 
  8320  67489 400555 CosmicComputer.txt
$ wc -l CosmicComputer.txt 
8320 CosmicComputer.txt
$ wc -w CosmicComputer.txt 
67489 CosmicComputer.txt
$ wc -c CosmicComputer.txt 
400555 CosmicComputer.txt
}}}

* diff
The ''diff'' command compares to files showing lines removed and then added to change the first argument to the second argument.
For example comparing readline.sh and readTokens.sh
{{{
$ diff read*
3,4c3,4
< while  read line; do
<    echo "I read: $line"
---
> while  read first second third; do
>    echo "I read: $first followed by $second followed by $third"
}}}
The ''-u'' flag shows the changes in context:

{{{
$ diff -u read*
--- readline.sh	2009-10-21 10:55:33.000000000 -0500
+++ readTokens.sh	2009-10-21 11:01:02.000000000 -0500
@@ -1,7 +1,7 @@
 #!/bin/bash
 echo -n "Type a line of tokens: "
-while  read line; do
-   echo "I read: $line"
+while  read first second third; do
+   echo "I read: $first followed by $second followed by $third"
    echo -n "More please: "
 done
 echo

}}}

* file 
The {{{file}}} command determines the type of a file using ''magic'' and other file system info (not the suffix). {{{man magic}}}
{{{
$ file *.xml
harry.xml:    XML  document text
Sketch12.xml: SVG Scalable Vector Graphics image
test2.xml:    XML  document text

$ file *.txt
aa.txt:             ASCII text
ab.txt:             ASCII text
a-new.txt:          empty
a.txt:              ASCII text
a..txt:             ASCII text
author.txt:         ASCII English text
az.txt:             ASCII text
aZ.txt:             ASCII text
bash-notes.txt:     ASCII text
blue.txt:           ASCII text
b.txt:              ASCII text
CosmicComputer.txt: ASCII English text
dos.txt:            ASCII text, with CRLF line terminators
error.txt:          ASCII text
err.txt:            ASCII text
Greek Words.txt:    UTF-8 Unicode text
new-file.txt:       empty
new-notes.txt:      ASCII text
no-tabs.txt:        ASCII English text
numbers.txt:        ASCII text
red.txt:            ASCII text
second-new.txt:     empty
sort-me.txt:        ASCII text
tabs.txt:           ASCII English text
temp2.txt:          ASCII text, with escape sequences
temp.txt:           ASCII text, with escape sequences
text.txt:           ASCII text
treesketch.txt:     PNG image, 466 x 530, 8-bit/color RGB, non-interlaced
t.txt:              ASCII text
}}}

* touch
Updates the access and modified times of a file; if the file doesn't exist it is created empty.

Things to remember about Linux file system
* file paths are composed with names separated by slashes {{{/}}}
* directories are just a special case of file
* file names are case sensitive: abc, Abc, aBc, abC, ABc, AbC, etc all different
* file names can be 256 characters; any character except slash {{{/}}} and null
* type suffixes are by convention only in Linux
* by convention names starting with period are treated as hidden files by most commands
* all files are in the same hierarchy even if they are in different file system/hardware/drive
* files are maintained by inode number which doesn't change when moved 
* every directory includes two links {{{.}}} and {{{..}}} that refer to the current directory and its parent.

There is a standard set of top level directories for Linux (may vary by distribution)
|/boot | contains boot up files |
|/bin | contains binaries for user commands needed for single user system |
|/usr/bin | binaries for user commands |
|/usr/local/bin | binary for locally installed applications |
|/sbin | binaries for system programs for single user system |
|/usr/sbin | binaries for system commands |
|/usr/local | locally installed program |
|/etc | configuration files for system |
|/lib | shared libraries |
|/usr/lib | shared libraries |
|/usr/local/lib | shared libraries |
|/mnt | mount points for externally mounted devices |
|/home | home directories for users |
|/root | home directory for super user |
|/var | spooling data from printers etc |
|/var/log | log files |
|/opt | data for applications |
|/tmp | tmp files |
|/sys | kernel data for plug'n'play devices |
|/dev | virtual directory of peripheral devices |
|/proc | virtual directory of os resource |
|/lost+found | data not-recovered after a crash |


# ls (list), stat             
# mkdir  (make directory)  
# pwd (present working directory) 
# cd (change directory)      
# cp (copy)                       
# mv (move)    
# rm (remove)                     
# cat (concat  files to stdout)
# more (view by page) or less         (less with -u for dos files)
# diff (differences)           
# head, tail (including tail -f)
# ln create symbolic link/alias for file
{{{
$ ln base.dat hard-link.dat # a hard link; new directory link for same inode
$ ln -s base.dat soft-link.dat # a symbolic ink; new directory link for path base.dat
$ ls -lh *.dat
-rw-r--r-- 2 steve steve 13 2009-10-19 10:16 base.dat
-rw-r--r-- 2 steve steve 13 2009-10-19 10:16 hard-link.dat
lrwxrwxrwx 1 steve steve  8 2009-10-19 10:18 soft-link.dat -> base.dat
$ stat -c %i base.dat  # display inode for base.dat
19808375
$ stat -c %i hard-link.dat 
19808375
stat -c %i soft-link.dat
19808375

$ cat base.dat 
This is file
$ cat hard-link.dat 
This is file
$ cat soft-link.dat 
This is file

$ mv base.dat old-base.dat
$ cat hard-link.dat 
This is file
$ cat soft-link.dat 
cat: soft-link.dat: No such file or directory

$ cat >base.dat
New file
$ cat soft-link.dat 
New file
$ cat hard-link.dat 
This is file
}}}
	
* every time you execute a command in bash which is not builtin (see {{{type}}}) a new process is //forked//.
* Putting a process into background with &, and stopped with ctl-z,checking it status, bringing it to the foreground.
* Process states, process id, job id; {{{ps}}} and {{{top}}}
* signals 
* {{{kill}}} and {{{kill -9}}} ({{{kill -s SIGTERM}}}  vs  {{{kill -s SIGKILL}}})
* {{{kill -s SIGCONT}}} and {{{kill -s SIGSTOP}}} with {{{xeyes&}}} as example
* process hierarchy
** {{{ps -F -C xeyes}}} print info for xeyes
** {{{ps -F ppid}}} for the parent Id from above
** {{{ps -F ppid}}} until we reach /sys/init the root process
* daemon processes
* running a process that survives the end of the session ({{{nohup}}})

With readline and echo you can write interactive scripts or process files a line at a time.
!!!! readline
The ''read'' command uses the Linux readline library to read a set of tokens from an input stream.  Each line of input is parsed (but no substitutions are performed_ and each token is assigned to an input variable until:
# all the input tokens are exhausted.  In which case, nulls are assigned to the remaining variables
# all the variables but one are exhausted.  In which case, the remainder of the line is assigned unparsed to the last variable.
{{{
#!/bin/bash
echo -n "Type a line of tokens: "
while  read first second third; do
   echo "I read: $first followed by $second followed by $third"
   echo -n "More please: "
done
echo
echo "All done."
}}}
{{{
$ ./readTokens.sh 
Type a line of tokens: One Two Three
I read: One followed by Two followed by Three
More please: One "Two Three"
I read: One followed by "Two followed by Three"
More please: One $PATH
I read: One followed by $PATH followed by 
More please: One `Date`
I read: One followed by `Date` followed by 
More please: One Two Three Four
I read: One followed by Two followed by Three Four
More please: One Two Three        Four
I read: One followed by Two followed by Three        Four
More please: One Two       Three Four
I read: One followed by Two followed by Three Four
More please: 
All done.
}}}
The input can also be redirected from a file:
{{{
$ ./readTokens.sh <donate.rec 
Type a line of tokens: I read: Bay followed by Ching followed by 500000 China
More please: I read: Cruella followed by Lumper followed by 725000 Malaysia
More please: I read: Harold followed by Earwicker followed by 1300050 Paraquay
More please: I read: Jack followed by Arta followed by 250000 Indonesia
More please: I read: More followed by Money followed by 2300000 Norway
More please: I read: Greedy followed by Guts followed by 023 Norway
More please: 
All done.
}}}
Where ''donate.rec'' contains blank separated fields:
{{{
$ cat donate.rec 
Bay Ching 500000 China
Cruella Lumper 725000 Malaysia
Harold Earwicker 1300050 Paraquay
Jack Arta 250000 Indonesia
More Money 2300000 Norway
Greedy Guts 023 Norway
}}}
The more common use of ''read'' is use a single argument which consumes the entire line:
{{{
#!/bin/bash
echo -n "Type a line of tokens: "
while  read line; do
   echo "I read: $line"
   echo -n "More please: "
done
echo
echo "All done."
}}}
!!!! select in
{{{
#!/bin/bash
select animal in cat dog bird hippopotamus
do
   echo $animal
   if [[ $animal == "hippopotamus" ]] ; then break; fi
done
echo "You picked hippo."
}}}
{{{
$ ./select-animal.bash 
1) cat
2) dog
3) bird
4) hippopotamus
#? 3
bird
#? 2
dog
#? 1
cat
#? 4
hippopotamus
You picked hippo.
}}}
The wildcard expressions used by Bash to match files are not regular expressions.  Sometimes they are referred to as ''glob'' expressions after a command that existed before wildcarding was built into Bash.  They are document in {{{man glob}}} or {{{info glob}}}

The regular expressions used by string processing functions like {{{sed}}} and {{{grep}}} are different and more powerful.  They are documented in {{{man regex}}} or {{{info regex}}}

Grep is a command which output lines matching the regex argument
{{{grep --color=auto a <donate.rec}}}
{{{grep --color=auto <donate.rec p}}}  

A Regular expression is composed of one or more branches separated by {{{|}}} which are componsed of pieces which are composed of atoms.

!!!!Atoms
* a character is an atom that matches that character
* the period ({{{.}}}) which matches any single character
* an ^ which matches the begining of the line
* a $ which matches the end of the line
* an escaped special character that matches special character (^.[$()|*+?{\) as if it were ordinary {{{grep --color=auto <CosmicComputer.txt '\$'}}}
* a bracket expression [acd]
** Simple {{{ [qx]}}} the characters list
** Range {{{[2-7]}}} characters in the range
** Combined simple and ranges concatenate {{{ [@0-3q]}}}
** Negated {{{ [^0-9a-zA-Z\ .,-]}}}
* A regular expression inclosed in parenthesis {{{ "([qv])"}}}; This is an extension
!!!!Piece
A piece is an atom optionally followed by
* ? -- Optional; this is an extension
* + -- One or more  {{{ [p@]+]}}}; this is an extension
* * -- Zero or more
* Or a braces containing a bound either {lower} or {lower,upper} for the number of times to repeat {{{ "[op]{2}"}}}; this is an extension

!!!!Branch
* A branch is one or more pieces concatenated {{{"[a-l][eo]+s" ==> p1=[a-l], p2=[eo]+, p3 = s}}}
* note surrounding a branch in parenthesis makes it an atom {{{"(o[bt]){2}"}}}

!!!!Regular expression 
* is an alternation of branches  {{{"(o[bt]){2}|s\-s"}}} ; This is an extension

There is also local based pre-defined classes which help with international charsets.  {{{[:lower:]}}} is a-z plus any diacriticals for the defined language locale.

!!! grep
The grep command is takes stdin applies a regular expression to each line and outputs lines with matches to stdout
Options:
* -E use extended syntax also use the command {{{egrep}}}
* -f <file> get pattern from file
* -i ignore case
* -v invert result
* -w match lines where the pattern matches a whole word
* -x only match lines that completely match the pattern

!!!sed
{{{sed}}} is a stream editor and applies basic editing commands to stdin and writes the results to stdout.
Example substitute 'grizzly' for 'robot'
{{{ sed s/robot/grizzly/ <CosmicComputer.txt  | egrep --color=auto robot }}}
{{{ sed s/robot/grizzly/g <CosmicComputer.txt  | egrep --color=auto robot }}}

The {{{g}}} flag applies substitution for all occurences of match in input

{{{ sed -n -e "18,28p" -e "200,208p" <CosmicComputer.txt }}}

The {{{-e}}} flag introduces a sed-command (not neccessary when there is only one.  The {{{-n}}} option tells the sed processor not to copy input to output without an explicit {{{p}}} command.

The {{{sed}}} command does not alter the input file. 

   (( from info sed -- *3.1 How 'sed' works ))
The sed command modifies lines from the specified File parameter  or standard input and writes them to standard output. The sed command includes many features for selecting lines to be modified and making changes only to the selected lines.

The sed command uses two work spaces for holding the line being modified: the pattern space, where the selected line is held; and the hold space, where a line can be stored temporarily.

An edit script consists of individual subcommands, each one on a separate line. The general form of sed subcommands is the following:

[address-range] function[modifiers]

The sed command processes each input file by reading an input line into a pattern space, applying all sed subcommands in sequence whose addresses select that line, and writing the pattern space to standard output. It then clears the pattern space and repeats this process for each line specified in the input File parameter. Some of the sed subcommands use a hold space to save all or part of the pattern space for subsequent retrieval.

When a command includes an address (either a line number or a search pattern), only the addressed line or lines are affected by the command. Otherwise, the command is applied to all lines. 
!!! Combining commands with pipelines
List all the sub directories of the current directory.
{{{stat -c %n=%F * | grep =directory | sed  s/=directory//g}}}

    
* Exercise write a command pipeline that lists all the suffixes used in the current directory
[[exercise-answer]]
!!! User id and permissions
Like most operating systems, you log in to Linux with a user name and password. Your permissions are determined by your userid and the ''groups'' you belong to.  
From the command line you can change your password with the {{{passwd}}} command.
You can view what groups you belong to with the {{{group}}} command.
You can change what user you are acting as with the {{{su}}} command.
You can see what users are logged in with the {{{users}}} and {{{who}}} command; who also reports how you are logged in as.  Note, who and user report the logged in user not who they are acting as.
!!! File permissions read, write, execute, etc
Every object that has a file path (most things in Linux) has a set of permissions.
The permissions are ''read'', ''write'' and ''execute''
The permissions applied to the ''owner'', members of ''group'' and ''other users''.

You can see the human readable form of the permission via the ''ls -l'' longform command for regular files, and stat -c %A for all files.
{{{
$ ls -l {aa,blue}.txt
-rw-r--r-- 1 steve steve  7 2009-10-07 15:40 aa.txt
-rw-rw-r-- 1 steve users 65 2009-10-13 12:14 blue.txt
$ stat -c %A {aa,blue}.txt
-rw-r--r--
-rw-rw-r--
$ ls -l local/
total 12
drwxr-xr-x 4 steve steve 4096 2009-09-18 14:37 int
drwxr-xr-x 3 steve steve 4096 2009-09-18 15:26 osa
drwxr-xr-x 3 steve steve 4096 2009-09-18 14:39 sps
$ stat -c %A local/
drwxr-xr-x
}}}
For ''ordinary files'':
| read | permision to read contents |
| write | permision to write contents |
| execute | permission to execute as a command |
For ''directories''
| read | permission to get directory listing |
| write | permission to add/delete entries |
| execute | permssion to change to directory as working directory |

We can alter the permissions on a file with ''chmod'' command:
{{{
chmod o-w blue.tx
$ ls -lh blue.txt 
-rw-r--r-- 1 steve users 12 2009-10-13 14:15 blue.txt
$ stat -c %A blue.txt 
-rw-r--r--
$ chmod a+w blue.txt # add write permission for all users 
$ stat -c %A blue.txt 
-rw-rw-rw-
$ chmod o-w blue.txt # take away write permission for other users
$ stat -c %A blue.txt 
-rw-rw-r--
}}}
So two files with different permission:
{{{
$ ls -lh {red,blue}.txt
-rw-rw-r-- 1 steve users 11 2009-10-13 13:11 blue.txt
-rw-r--r-- 1 steve users 25 2009-10-13 13:42 red.txt
}}}
If I'm logged in a ''george'' a member of group ''users''
{{{
$ groups
users
$ cat >blue.txt
This works.
$ cat >red.txt
bash: red.txt: Permission denied
}}}

Note: there is also an octal form of the permissions with a octal digit for each permission group (owner, group, other).
{{{
$ stat -c %A {red,blue}.txt
-rw-r--r--
-rw-rw-r--
steve@Twisty:~/lnfsm$ stat -c %a {red,blue}.txt
644
664
}}}

The ''umask'' command lets you change the default permissions for newly created files.
{{{
$ umask -S
u=rwx,g=rx,o=rx
steve@Twisty:~/lnfsm$ umask -p
umask 0022

$ umask 077
$ umask -S
u=rwx,g=,o=

$ touch a-new.txt
$ ls -lh a-new.txt 
-rw------- 1 steve steve 0 2009-10-13 14:48 a-new.txt
$ rm a-new.txt 
$ umask 020
$ umask -S
u=rwx,g=rx,o=rwx
$ umask 027
$ umask 007
$ umask -S
u=rwx,g=rwx,o=
$ touch a-new.txt
$ ls -lh a-new.txt 
-rw-rw---- 1 steve steve 0 2009-10-13 14:49 a-new.txt
$ rm a-new.txt 
$ umask 022
$ touch a-new.txt
$ ls -lh a-new.txt 
-rw-r--r-- 1 steve steve 0 2009-10-13 14:50 a-new.txt

}}}
You can use the ''umask'' command in the bash run command (.bashrc) to set your own default.
      
!!! Account properties-home directory, shell command
The properties of an account are stored in the file /etc/passwd.  The line for the ''george'' account is:
{{{
george:x:1001:100:Another user,,,,:/home/george:/bin/bash
}}}
This gives (in order): loginid, delete password, user id, group id, full name of user, (some extra fields email, phone # etc), home directory and shell
there are commands that let you change attributes.  For example, ''chsh'' changes your default shell progam.  
!!! Super-user (root), sudo and sudoes
Every Linux system has a distinquished user ''root'' called the super-user that has all-permissions to all files.  Well run systems do not allow users to login as root and most system administrators carefully protect the root password.

Typically administrative tasks are performed by commands stored in {{{ /sbin, /usr/sbin, and /usr/local/sbin}}}.  In order, allow users to perform some of these commands the command ''sudo'' executes it's arguments as a command with root permissions.  This command is managed through a file /etc/sudoers
{{{
$ ls -lh t.txt 
-rw-r--r-- 1 steve steve    5 2009-10-05 15:59 t.txt
$ chown steve:users t.txt 
chown: changing ownership of `t.txt': Operation not permitted
$ sudo chown steve:users t.txt
[sudo] password for steve: 
$ ls -lh t.txt 
-rw-r--r-- 1 steve users 5 2009-10-05 15:59 t.txt
}}}
If the sudoers file does not permit sudoing this command:
{{{
$ chmod a+w blue.txt 
chmod: changing permissions of `blue.txt': Operation not permitted
george@Twisty:/home/steve/lnfsm$ sudo chown steve:steve t.txt 
[sudo] password for george: 
george is not in the sudoers file.  This incident will be reported.
}}}
From the auth log
{{{
sudo:   george : user NOT in sudoers ; TTY=pts/2 ; PWD=/home/steve/lnfsm ; USER=root ; COMMAND=/bin/chown steve:steve t.txt
}}}


!!!Logging in from Windows: SSH and PuTTY
http://www.chiark.greenend.org.uk/~sgtatham/putty/

!!!File Transfers WinSCP, SFTP, SCP and FTP
http://winscp.net/eng/index.php

!!!SSH between servers
Telnet and ftp vs ssh and sftp
Public Key Encryption 
Establing an ssh session between a client and a server machine:
# Client connects to server and requests ssh session
# Server sends the public half of its key pair to server
# If client has previously connected with server it compares keys and halts on difference; otherwise it offers to save key
# Client creates random-symmetric session key and encrypts it with the server's public key
# Server initiates communication using the session key and challenges user with for either password or already established client public key.
How to setup SSH keys between the icpavmg51 and icpavmg52 Linux Servers
page 3
!!! Local Editing Crimson
http://www.crimsoneditor.com/
!!! Local testing of scripts CygWIN
http://www.cygwin.com/
!!! Remote GUI's X Windows and VNC
http://x.cygwin.com/
http://www.tightvnc.com/