Author: robertsgreibers

5 Secret Debugging Tools For Python And Django

Alright, Roberts Greibers here with another experience-based Python post.

I’ve been working professionally as a Python developer for more than 5 years and now I’m also helping people who are at the start of their careers (or in a similar position I was in) to become Python developers. 

I used to be a quality assurance engineer – manually tested website projects and mobile apps. (iOS, Android)

So I totally get how it is when you want to move away from your current role and get into a more profitable position, for example, become a Python/Django developer.  

What I’m about to show you in this post are five of the most important tools I used and my clients are using now (see a cut from a LIVE Zoom video call below) that made me look like a senior Python developer even though I was just starting to learn Python.

Continue reading

Amazon Scraping In Python Selenium

Scraping data from any kind of website with Python can be exciting and challenging at the same time. Also, you can learn a lot about development and writing simple scripts – practice Python’s fundamentals.

More than 5 years ago, when I first started to get to know what Python is capable of – the very first thing I wanted to learn was how to automate browser actions – open websites, click on buttons, enter text into input fields, etc.

Back then I was learning how to automate facebook.com

Within the following steps, you will learn how to automate scraping data from amazon.com from A-Z.

I’ve prepared steps with everything you need to do in order to be able to fully automate the process and gather data for up to 100 products in less than a minute.

With the provided code you’ll have a good base for starting the Amazon website automation and you’ll be able to expand and extend my code to whatever your goals are.

Follow the steps below.

Continue reading

Python Regex Groups, Match and Search

Hey, Roberts Greibers here. I’ll give you a brief intro about my 7-year experience as a Python developer down the line – later in this post. 

But now I want you to scroll a bit more and dive right into Python regex groups and the Python regex example I’ve greatly detailed below. 

The following regex Python example is going to show you.. ⚠️

  • What actually is import re in Python and how you can use it!
  • How to THINK and come up with your own Python regex patterns
  • A real-life Python regex example (sent in by a reader of this blog post)

If you’ve been Google searching for Python re examples, you’ve found the RIGHT place!

Most regular expression Python examples you’ll find online are very theoretical, showing you just the concept of a regex capturing group.. but they don’t tell you how to THINK in order to come up with your own solutions – your own regex patterns.

In this post, I’ll take a real Python regex situation and explain what fundamental steps I take to come up with a solution. πŸš€

And if you really think about it…

It’s the ONLY way for you to truly understand regex Python examples and use them for your benefit. You have to understand the fundamental process behind solving a Python problem.

So, if getting deeper into a real regular expression Python example sounds interesting to you – keep on reading!

I’m explaining the whole story of how I came to the following situation in Python where regex capture group was part of the solution down below! πŸ‘‡πŸ»

Continue reading

Log File Parsing In Python

In this tutorial, you will learn how to open a log file, read a log file, and create a log file parser in Python, essentially building a so-called “Python log reader”.

To open a log file in Python, read a log file, and actually parse a log file or any type of text file in order to extract specific information is not that hard if you know a bit of Python log file parsing and regex pattern match usage.

Python itself is a perfect tool to open a log file for parsing and it does not require any third-party modules. Believe me, the first thing I did was a Google search for a “Python log reader” and “Python file parsing” a couple of years ago when I first started to work on parsing text files in Python. Ever since those days, I’ve learned to work with Python and regex very efficiently, see more details in a recent post I did about how to parse XML SOAP response with Python and regex by clicking here.

In my day job, a while back I was working on testing Skype for Business iOS application as a test engineer and it came to the point where I had to open and manually collect the SfB iOS application log files in order to see all HTTP requests and received HTTP responses.

Ever since then I’ve switched to backend development with Python/Django and also helped people to go into a similar path. See a cut from a recent coaching call here. And more about client testimonials here.

Anyways, in this specific situation, I had to figure out a good way to open iOS log files and parse them to search a log file for properties like:

<code><property name="saveMessagingHistory">Enabled</property></code>

Usually, properties were buried under a bunch of other not-so-important log file dumps, for example:

INFO UTILITIES /Volumes/ServerHD2/buildagent/workspace/200615/client_ios_sfb/dependencies/client-shared_framework_sfbplatform/src/dev/lyncMobile/platform/tracing/privateIos/CMTrace.mm/173:Version Information 6.12.0.65
2016-12-20 13:16:52.303 SfB[417:1af74bc40] INFO UTILITIES CTimer.cpp:657 TimerMap is created 
2016-12-20 13:16:52.342 SfB[417:1af74bc40] INFO UI SFBAppDelegate.mm:69 Application will finish launching with options
2016-12-20 13:16:52.345 SfB[417:1af74bc40] INFO UTILITIES CStorageManager.mm:146 Creating StorageManager
2016-12-20 13:16:52.347 SfB[417:1af74bc40] INFO UTILITIES CStorageManager.mm:187 Initializing StorageManager
2016-12-20 13:16:52.357 SfB[417:1af74bc40] INFO APPLICATION CApplication.cpp:3400 Initialize Internal Begin
2016-12-20 13:16:52.361 SfB[417:1af74bc40] INFO UTILITIES COsInformation.mm:398 User UI language identifier en was mapped to en-US 1033
2016-12-20 13:16:52.362 SfB[417:1af74bc40] INFO UTILITIES COsInformation.mm:106 Device Version Info - Model=iPhone, HardwareModel=iPhone9,3, SystemName=iOS, SystemVersion=10.1
2016-12-20 13:16:52.362 SfB[417:1af74bc40] VERBOSE APPLICATION CApplication.cpp:3415 Initialize Internal -- App State Query Established
2016-12-20 13:16:52.363 SfB[417:1af74bc40] INFO UTILITIES CNetworkMonitor.cpp:70 Successfully started listening to network events
2016-12-20 13:16:52.364 SfB[417:1af74bc40] INFO UTILITIES CNetworkMonitor.cpp:229 Reachabilility Flags IsWWAN(0):Reachable(1):TransientConnection(0):ConnectionRequired(0):ConnectionOnTraffic(0):InterventionRequired(0):ConnectionOnDemand(0):IsLocalAddress(0):IsDirect(0)
2016-12-20 13:16:52.364 SfB[417:1af74bc40] INFO UTILITIES CNetworkMonitor.cpp:198 Updated networkAvailableToConnect(NoNetwork) -> WiFi, isInAirplaneMode(0) -> 0
2016-12-20 13:16:52.364 SfB[417:1af74bc40] INFO UTILITIES CTimer.cpp:227 Created timer instance (0x70286428) for runloop (0x7017e780)

While I was searching for specific properties in those log files, I realized it’s going to be really time-consuming to go through everything manually.

In order to save time, I had to come up with a good way to use Python file parsing, and long behold I managed to write code for a log file parsing Python script.

It’s a very simple way of searching a log file with Python.

If you want to test the following Python log parsing script with a similar text file, you have to download Skype for Business iOS log file here:

Continue reading