[SOLVED] Problem using AWK in shell script

[MoonRise]

So, title says it all. The code is

awk -F $'\t' '$6 != ""' $myFl >> $myFlf

If I run that in Konsole where $myFl and $myFlf are named Input.txt and InputF.txt (

awk -F $'\t' '$6 != ""' Input.txt >> InputF.txt

) it produces the file expected.

When I have that exact same code but where the file names are variable as shown above I receive a 0kb file. Empty. Nada. Nothing.

I can't for the life of me figure why and no level of searching has revealed a clue.

Anyone!! Please Help!!

[jlittle]

Well, it is inexplicable on the face of it. My first thought is that the file and variable names are confusing; mixing f and F, and mixed case generally, I find confusing. Depending on your font the difference between ell, "l", and one, "1", can be treacherous.

I suggest running

echo "@$myFl@  @$myFlf@"

to be sure the variables really do have the values you think, and that you check that the current directory is the one with the right Input.txt.

[MoonRise]

I actually did that and truly I can see the script touch the files as I have that folder up. It's really odd that it works alone in konsole but not in the script. I just have no clue.

Have a thought and will try that but if no go, I'm stumped.

[MoonRise]

OK, so I think I might know the issue and really have no clue how to fix.

This is a script and variables start off with a "$".

See my code:

awk -F $'\t' '$6 != ""' $myFl >> $myFlf

See the "$6"? It is being interpreted as a variable and therefore comes up empty. AWK in konsole knows how to interpret that (Column 6) because it isn't script based and not interpreted as variable. In the script it is and therefore is blank. So, what to do!?!?

I'm so fried now I can't think of how to force it to represent itself as is.

[Snowhog]

Maybe see How to use a variable inside an awk statement?

[MoonRise]

Nope. I'm close. It has something to do with that whole expression and how a script interprets it over how konsole does. I'm too tired to figure it right now. This one has beat me!

[Snowhog]

How To Use awk In Bash Scripting

I'm almost certain that the above contains the answer you are missing. Scroll down to Awk in Shell Scripts – Passing Shell Variables TO Awk

[MoonRise]

Thing is I'm not passing variables into awk. It is the filter condition, I think, that is the issue. I'm looking for Column 6 entries that are not blank and trying to pass those back out to a new file so I can get rid of entries where column 6 has no value.

As I said, tired right now. I'll re-read later.

Thanks though!!

[Snowhog]

Instead of trying to script it in bash, maybe write a function?

[oshunluvr]

Run awk outside of the bash script?

You can put all awk commands in a file and call the same from a shell script using the following syntax:
awk -f mypgoram.awk input.txt

[jlittle]

awk -F $'\t' '$6 != ""' $myFl >> $myFlf

See the "$6"? It is being interpreted as a variable and therefore comes up empty.

No, because the $6 is within single quotes the shell shouldn't touch it. Only if there's a double evaluation would that happen.

In a konsole I presume you're running bash. In the script, is there a "shebang" line? That's the first line beginning with #!, like

#!/bin/bash

How are you invoking the script? Is it possible for us to see it?

Regards, John Little

[MoonRise]

By all means. And yes to the first the question. See below. All works great except that. What is confounding me is the "awk" portion works in konsole not this script, unless it is how I have the "#!/bin/bash"

#! /bin/sh
#

cd /home/moonrise/Documents/My_Work/fakeServer/delivEDI/dropEDI
for file in /home/moonrise/Documents/My_Work/fakeServer/delivEDI/dropEDI/*; do
myDATE=`date '+%Y%m%d'`
myTIME=`date '+%H%M%S'`
myFl="EDI_file_"$myDATE$myTIME".TXT"
myFlf="EDI_file_f_"$myDATE$myTIME".TXT"
mv ${file##*/} "/home/moonrise/Documents/My_Work/fakeServer/delivEDI/dropEDI/$myFl"
echo $myFl
sed -i '/BLINDED/d' $myFl
done
awk -F $'\t' '{if($6 != "") {print}}' $myFl >> $myFlf

[MoonRise]

Do appreciate the help on this. For some reason I cannot wrap my head around this one. Could be the headache I have which I really think was brought on by this.

[Snowhog]

Maybe.... https://superuser.com/questions/7958...-a-bash-script

[MoonRise]

Maybe... https://superuser.com/questions/7958...-a-bash-script

Thanks but no go there either. It is probably just me. I understand what AWK is and it does work, outside of the script. Just can't figure how to get it to run as part of the script. Oh well, was worth the effort. Learned quite a bit.