How to Modify a File Using SED

Let’s take a look at a microchip project that was found to cause issues in the past. Upon trying to build the problematic example it was found that one of the Makefiles generated throughout the use of the Makefile generator command, “prjMakefilesGenerator“ contained invalid paths. Rather than modify the make file manually every time the Makefile generator command was called, it was decided that SED could be used to automate the process.

prjMakefilesGenerator -v <PATH_TO_MICROCHIP_PROJECT>

# Example
PROJ_PATH="projects/microchip/same70_xult/mplab/aws_demos/firmware/aws_demos.X"
prjMakefilesGenerator -v ${PROJ_PATH

The following commands are what were used to attempt to build the project.

pushd ${PROJ_PATH}
make clean
make all
popd

Those commands then produced the following error messages indicating the build failed.

User defined post-build step: [/opt/microchip/xc32/v2.41/bin\\xc32-objcopy -p -I ihex -O binary dist/same70_xult_nvm/production/aws_demos.X.production.hex dist/same70_xult_nvm/production/aws_demos.X.production.bin]
/bin/sh: 1: /opt/microchip/xc32/v2.41/binxc32-objcopy: not found
nbproject/Makefile-same70_xult_nvm.mk:108: recipe for target '.build-conf' failed
make[2]: *** [.build-conf] Error 127
make[2]: Leaving directory '/microchip-mqtt/projects/microchip/same70_xult/mplab/aws_demos/firmware/aws_demos.X'
nbproject/Makefile-impl.mk:39: recipe for target '.build-impl' failed
make[1]: *** [.build-impl] Error 2
make[1]: Leaving directory '/microchip-mqtt/projects/microchip/same70_xult/mplab/aws_demos/firmware/aws_demos.X'
nbproject/Makefile-impl.mk:54: recipe for target '.all-impl' failed
make: *** [.all-impl] Error 2

From the first line of the log, it can be seen that the path to xc32-objcopy was not correctly converted to forward slash. Fortunately, SED was able to be leveraged to automatically fix this problem in future attempts to build the project. The SED command is a stream editor that can perform insertion, deletion, and substitution of text. What was needed from SED was essentially to search for the incorrect path and replace it with the valid path.

# sed command for substituing string
$sed "s/<SEARCHING_PATTERN>/<REPLACEMENT_STRING>/" <PATH_TO_THE_FILE>

# Example
$cat example.txt
This file has invalid paths

$ sed "s/invalid/valid/" example.txt
This file has valid paths

The s character in the example above demonstrates the substitution operation. The forward slashes / are used as delimiters in the replacement command. When running the command above, it would only print out the content with the substituted string. To modify the file itself, the -i option is needed for in-place substitution.

if [[ -e ${FILE_TO_BE_MODIFIED} ]]; then
    sed -i "s/<SEARCHING_PATTERN>/<REPLACEMENT_STRING>/g" ${FILE_TO_BE_MODIFIED}
fi

Before these concepts could be applied to the problematic build commands, a few more things were needed. First was to check if the file that needed to be modified existed, and if it did, then the file would be modified in-place. Secondly the g character at the end of the string represents a global replacement. In this case, the desired change was to change the back-slashes in the Makefile to forward-slashes. Finally, the completed SED command could be pasted above the make command as seen below. In doing so, all subsequent attempts to build the project were successful.

PROJ_PATH="projects/microchip/same70_xult/mplab/aws_demos/firmware/aws_demos.X"
PROJ_CONF="same70_xult_nvm"

pushd ${PROJ_PATH}
if [[ -e nbproject/Makefile-${PROJ_CONF}.mk ]]; then
    sed -i "s/\\\\\\/\\//g" nbproject/Makefile-${PROJ_CONF}.mk
fi
make clean
make all
popd

SED is a powerful tool that was able to assist in solving the build errors described above. Aside from this example, there may be numerous other times when files or configs may need to be modified such as a Doxyfile output directory and project name. In those cases, consider using SED as it will most definitely come in handy.

Alternatives to SED

Is SED not your style? Not to worry, there are many other ways to accomplish the same thing. Tools like awk, may be used to accomplish a similar task to what was demonstrated above, and if you are more familiar with some other tools you should still be able to incorporate what was discussed above.

What about Windows?

While Windows may be lacking some of the tools found in a UNIX based system, many people online have had success using features of powershell to act in a similar way to SED. Also consider installing WSL to get access to a CMD-line interface that may be interacted with as if it was a Linux system.

Next Steps

We have talented engineers on hand ready to help you with all aspects of your EmbedOps journey. You can Book a Call with us to discuss your embedded project, or sign up for free for our EmbedOps CI and build management product.