# AWK and SED Examples

These examples explore the combined use of AWK and SED for text processing tasks. 

## Example 1: AWK and SED for Text Transformation

### Input File (input.txt)

```
Carl Sanchez,25,Engineer
Jane Smith,30,Designer
Bob Johnson,22,Developer
```

### Goal

We want to transform the CSV data into a formatted table with headers.

### Solution

```bash
awk -F',' '{printf "%-15s %-3s %-10s\n", $1, $2, $3}' input.txt | sed '1i\
Name            Age Job\
--------------- --- ----------'
```

### Output

```
Name            Age Job
--------------- --- ----------
Carl Sanchez    25  Engineer
Jane Smith      30  Designer
Bob Johnson     22  Developer
```

## Example 2: AWK and SED for Selective Printing

### Input File (input.txt)

```
apple
banana
cherry
date
```

### Goal

Print lines containing "c" or "r" using AWK and SED.

### Solution

```bash
awk '/c|r/' input.txt | sed -n '/c\|r/p'
```

### Output

```
cherry
```

## Example 3: AWK and SED for Filtering and Sorting

### Input File (input.txt)

```
apple,5
banana,2
cherry,8
date,1
```

### Goal

Filter lines with the second column greater than 2, sort by the second column, and add a header.

### Solution

```bash
awk -F',' '$2 > 2' input.txt | sort -t',' -k2n | sed '1i\
Fruit,Quantity'
```

### Output

```
Fruit,Quantity
apple,5
cherry,8
```

## Example 4: AWK and SED for Text Extraction

### Input File (input.txt)

```
Lorem ipsum dolor sit amet, consectetur adipiscing elit.
Sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris.
```

### Goal

Extract sentences containing the word "sed" and replace "sed" with "AWK".

### Solution

```bash
awk 'BEGIN{RS="\\."} /sed/' input.txt | sed 's/sed/AWK/g'
```

### Output

```
AWK do eiusmod tempor incididunt ut labore et dolore magna aliqua
```

## Example 5: AWK and SED for HTML Tag Extraction

### Input File (input.html)

```html
<div class="container">
    <p>This is a paragraph.</p>
    <a href="https://example.com">Visit our website</a>
</div>
```

### Goal

Extract and list all HTML tags present in the file.

### Solution

```bash
sed -n 's/<[^>]*>//p' input.html | awk '{print tolower($0)}'
```

### Output

```
div class="container"
p
/p
a href="https://example.com"
/a
/div
```

## Example 6: AWK and SED for JSON Formatting

### Input File (input.json)

```json
{
    "name": "Carl Sanchez",
    "age": 30,
    "city": "Indianapolis"
}
```

### Goal

Format the JSON to be human-readable using AWK and SED.

### Solution

```bash
awk -v RS= '{$1=$1}1' input.json | sed 's/":"/": "/g; s/{/{\n/g; s/}/\n}/g'
```

### Output

```
{
    "name": "Carl Sanchez",
    "age": 30,
    "city": "Indianapolis"
}
```

## Example 7: AWK and SED for Log Analysis

### Input File (logfile.txt)

```
2023-01-15 10:23:15 INFO User login successful
2023-01-15 10:30:42 ERROR Invalid password attempt
2023-01-15 11:05:18 INFO User logout
```

### Goal

Extract and format log entries containing "ERROR".

### Solution

```bash
awk '/ERROR/' logfile.txt | sed 's/\([0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\} [0-9]\{2\}:[0-9]\{2\}:[0-9]\{2\}\) \(ERROR .*\)/Timestamp: \1\n\2\n/'
```

### Output

```
Timestamp: 2023-01-15 10:30:42
ERROR Invalid password attempt
```

## Example 8: AWK and SED for CSV Transformation

### Input File (data.csv)

```
Name,Age,Occupation
John,28,Engineer
Jane,35,Designer
Bob,22,Developer
```

### Goal

Transform the CSV data into a tab-separated format.

### Solution

```bash
awk -F',' '{printf "%s\t%s\t%s\n", $1, $2, $3}' data.csv | sed '1i\
Name\tAge\tOccupation'
```

### Output

```
Name    Age     Occupation
John    28      Engineer
Jane    35      Designer
Bob     22      Developer
```

## Example 9: AWK and SED for XML Tag Removal

### Input File (data.xml)

```xml
<root>
    <item>Apple</item>
    <item>Banana</item>
    <item>Cherry</item>
</root>
```

### Goal

Remove all XML tags and extract the content.

### Solution

```bash
sed 's/<[^>]*>//g' data.xml | awk 'NF'
```

### Output

```
Apple
Banana
Cherry
```

## Example 10: AWK and SED for Markdown Table Conversion

### Input File (table.md)

```markdown
| Name  | Age | Occupation  |
|-------|-----|-------------|
| John  | 28  | Engineer    |
| Jane  | 35  | Designer    |
| Bob   | 22  | Developer   |
```

### Goal

Convert the Markdown table into a CSV format.

### Solution

```bash
sed -n '/|/p' table.md | awk -F'|' '{gsub(/^[ \t]+|[ \t]+$/, "", $2); gsub(/^[ \t]+|[ \t]+$/, "", $3); gsub(/^[ \t]+|[ \t]+$/, "", $4); printf "%s,%s,%s\n", $2, $3, $4}'
```

### Output

```
Name,Age,Occupation
John,28,Engineer
Jane,35,Designer
Bob,22,Developer
```

## Example 11: AWK and SED for HTML List Extraction

### Input File (list.html)

```html
<ul>
    <li>Apple</li>
    <li>Banana</li>
    <li>Cherry</li>
</ul>
```

### Goal

Extract and list items from an HTML unordered list.

### Solution

```bash
sed -n '/<li>/s/<li>\(.*\)<\/li>/\1/p' list.html | awk '{print "- " $0}'
```

### Output

```
- Apple
- Banana
- Cherry
```

## Example 12: AWK and SED for YAML to JSON Conversion

### Input File (data.yaml)

```yaml
name: Carl Sanchez
age: 30
city: Indianapolis
```

### Goal

Convert YAML data into JSON format.

### Solution

```bash
sed -n '/^[^#]/s/: /: "/p' data.yaml | awk '{printf "%s", $0} END {print "\""}' | sed 's/"/\\"/g' | awk 'BEGIN {print "{"} {print "  " $0} END {print "}"}'
```

### Output

```
{
  name: "Carl Sanchez",
  age: "30",
  city: "Indianapolis"
}
```

## Example 13: AWK and SED for Extracting URLs

### Input File (text_with_urls.txt)

```
Visit our website: https://example.com
For more information, check out: https://documentation.com
```

### Goal

Extract and list all URLs from the text.

### Solution

```bash
sed -n 's/.*\(https\?:\/\/[^ ]*\).*/\1/p' text_with_urls.txt | awk '!a[$0]++'
```

### Output

```
https://example.com
https://documentation.com
```

## Example 14: AWK and SED for Data Filtering and Sorting

### Input File (data.txt)

```
Name|Age|Occupation
John|28|Engineer
Jane|35|Designer
Bob|22|Developer
```

### Goal

Filter entries with Age greater than 25 and sort by Age.

### Solution

```bash
awk -F'|' '$2 > 25' data.txt | sort -t'|' -k2n | sed '1i\
Name|Age|Occupation'
```

### Output

```
Name|Age|Occupation
Jane|35|Designer
John|28|Engineer
```

## Example 15: AWK and SED for Line Reversal

### Input File (reverse_me.txt)

```
Line 1
Line 2
Line 3
```

### Goal

Reverse the order of lines in the file.

### Solution

```bash
awk '{a[i++]=$0} END {for (j=i-1; j>=0;) print a[j--] }' reverse_me.txt | sed '1i\
Reversed Lines'
```

### Output

```
Reversed Lines
Line 3
Line 2
Line 1
```

## Example 16: AWK and SED for Word Frequency Analysis

### Input File (text_analysis.txt)

```
apple orange banana
banana apple orange
orange banana apple
```

### Goal

Count the frequency of each word and sort the results.

### Solution

```bash
awk '{for(i=1; i<=NF; i++) words[$i]++} END {for(word in words) print word, words[word]}' text_analysis.txt | sort -k2n -r | sed '1i\
Word Frequency'
```

### Output

```
Word Frequency
banana 3
apple 3
orange 3
```

## Example 17: AWK and SED for JSON Pretty Printing

### Input File (json_data.json)

```json
{"name":"John","age":30,"city":"Indianapolis"}
```

### Goal

Format the JSON data for better readability.

### Solution

```bash
awk -v RS= '{$1=$1}1' json_data.json | sed 's/":"/": "/g; s/{/{\n/g; s/}/\n}/g'
```

### Output

```
{
  "name": "John",
  "age": 30,
  "city": "Indianapolis"
}
```

## Example 18: AWK and SED for Extracting Email Addresses

### Input File (emails.txt)

```
Contact us at: info@example.com
For support, email support@example.com
```

### Goal

Extract and list all email addresses from the text.

### Solution

```bash
sed -n 's/.*\b\([a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}\)\b.*/\1/p' emails.txt | awk '!a[$0]++' | sed '1i\
Email Addresses'
```

### Output

```
Email Addresses
info@example.com
support@example.com
```

## Example 19: Convert Markdown Headings to HTML

### Input (markdown_headings.md)

```markdown
# Heading 1
## Heading 2
### Heading 3
```

### Goal

Convert Markdown headings to HTML headings.

### Solution

```bash
sed -n 's/^# \(.*\)$/\<h1\>\1\<\/h1\>/p; s/^## \(.*\)$/\<h2\>\1\<\/h2\>/p; s/^### \(.*\)$/\<h3\>\1\<\/h3\>/p' markdown_headings.md
```

### Output

```html
<h1>Heading 1</h1>
<h2>Heading 2</h2>
<h3>Heading 3</h3>
```

## Example 20: Convert Markdown Lists to HTML

### Input (markdown_lists.md)

```markdown
- Item 1
- Item 2
  - Subitem A
  - Subitem B
```

### Goal

Convert Markdown lists to HTML lists.

### Solution

```bash
awk '/^-/ {gsub(/^-/, "<li>"); print "<ul>"; print $0} !/^-/ {gsub(/^ /, ""); print $0 "</li>"} END {print "</ul>"}' markdown_lists.md
```

### Output

```html
<ul>
<li>Item 1</li>
<li>Item 2<ul><li>Subitem A</li><li>Subitem B</li></ul></li>
</ul>
```

## Example 21: Convert Bold and Italic Markdown to HTML

### Input (markdown_bold_italics.md)

```markdown
**Bold Text** *Italic Text*
```

### Goal

Convert Markdown bold and italic text to HTML.

### Solution

```bash
sed -e 's/\*\*\(.*\)\*\*/\<strong\>\1\<\/strong\>/g' -e 's/\*\([^*]*\)\*/\<em\>\1\<\/em\>/g' markdown_bold_italics.md
```

### Output

```html
<strong>Bold Text</strong> <em>Italic Text</em>
```

## Example 22: Reverse the Order of Lines

### Input (original_order.txt)

```
Line 1
Line 2
Line 3
```

### Goal

Reverse the order of lines in the file.

### Solution

```bash
awk '{a[i++]=$0} END {for (j=i-1; j>=0;) print a[j--] }' original_order.txt
```

### Output

```
Line 3
Line 2
Line 1
```

## Example 23: Sort Lines Alphabetically

### Input (unsorted_lines.txt)

```
Banana
Apple
Cherry
```

### Goal

Sort lines alphabetically.

### Solution

```bash
sort unsorted_lines.txt
```

### Output

```
Apple
Banana
Cherry
```

## Example 24: Sort Lines by Line Length

### Input (lines_by_length.txt)

```
Short
VeryLongLineWithManyCharacters
MediumLength
```

### Goal

Sort lines by their length.

### Solution

```bash
awk '{ print length, $0 | "sort -n"}' lines_by_length.txt | cut -d ' ' -f2-
```

### Output

```
Short
MediumLength
VeryLongLineWithManyCharacters
```