Python: new and old things

Python logo

There are several things that need to be bolded for every py programmer to be used or to have information on how and why is used.

Type hints

Python 3.5 brings type hints for clear function/interface usage. Idea is to set proper input instead of sending different type variables. Example of bad code:

def greeting(name):
    return 'Hello ' + name

Here we can pass numbers, boolean, or another complex object (class, function) and produce issue.

The clean version goes:

def greeting(name: str) -> str:
    return 'Hello ' + name

If you have 1000 lines inside of your codebase, for sure you don't understand this approach. But if your code goes up to 10 million lines of code - this seems really good practice.

Assignment expressions

In version Python 3.8 you have assignment expressions where is possible to use ":=" as part of a larger expression.

if (n := len(a)) > 10:
    print(f"List is too long ({n} elements, expected <= 10)")

This is good and not good. Let me explain why. A a person who lives in Py world, this lead me to think what would do programmer that don't take time to make clean code. Example of growing expression:

if (n := len(c := (d := a.replace("a","g"))[:len(a)])) > 10:
    print(f"List is too long ({n} elements, expected <= 10)")

Clean code could be:

a = "asdasdasdasdasdasd"
d = a.replace("a","g") 
c = d[:len(d)]

if (n := len(c)) > 10:
    print(f"List is too long ({n} elements, expected <= 10)")

This is why a new approach could lead to messy code.

range() and xrange() in Py2.x

People still using Python 2.x. Some of the usages are in web services. Sometimes they need to produce a bunch of IDs from 1 to 100 million. One approach is to use range() which return list. xrange() return generator object. What is the difference between them? In the first case, range, it uses large amounts of memory to set all numbers (integer, 4 bytes) into memory. So 100 000 000 x 4 is a lot of memory. Generator object objects that produce next() value and using just one size of integer for memory. This is used for example subnet generator, and picking up the second IP from the list. So imagine if you have 1000 subnets to generate each time with range() and xrange(). This happened before and I saw the issue. Change from range() to xrange() brings better work with memory and stopped to crashing web services.

New Dictionary algorithm

In Python 3.6 is introduced a new algorithm for dict objects. Proposed by Raymond Hettinger on Py Dev list New Dict proposal it brings between 20% and 25% smaller compared to Python 3.5. Good video on this topic is Raymond Hettinger Modern Python Dictionaries

asyncio — Asynchronous I/O, event loop, coroutines and tasks

It was introduced in Python 3.4. From the official documentation: "This module provides the infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives" Asyncio usage

Curenje kljuceva za info bot ministarstva za sajber sigurnost i jos 3 stvari koje nisam upamtio

U slobodno vrijeme kada nemam sta da radim sasvim slucajno sam naletio na github repo (tamo gdje ostavljamo kod za publikaciju, da se koristi od strane drugih ljudi) ministra Rajcevica.

Sve bi bilo super da unutar tog koda nije pobrisana baza sa pravim kljucevima od bota koji je trebao da sluzi za raniju verziju CERT-vog profila gdje bi taj bot kacio vijesti i informacije na tviter. Github sve pobrisane fajlove cuva kao dio baze koda. Tako da je nemoguce pobrisati bilo sta - sem ako se kompletan repo pobrise sa githuba.

github repo bota

Nakon 3 min pregleda koda dosao sam do zakljucka da postoji baza sa kljucevima koja je pobrisana. Prva pomisao mi je bila da pripada profilu ministra jer je pisalo njegovo tviter nalog ime. Prvo sto sam uradio jeste poslao CERTu da kljuceve pobrisu i da se uradi tzv revoke (znaci da se obnove novi kljucevi dok se stari u ovom slucaju za koje znamo odbacuju za dalje koriscenje).

CERT email

Isti dan su naveli da je slucaj zatvoren. Sacekao sam 24h. sljedeci dan sam shvatio da kljucevi nisu ponisteni i da se i dalje u upotrebi.

Pregledom starog bloga g ministra Rajcevica naisao sam da su kljucevi zapravo za bot a ne njegov profil.

Blog arhiva Rajcevic

Tu sam nasao da je trebalo da bude kao neki bot koji bi se koristio u ex-CERT RS gdje bi isti kacio na tviter informacije i slicno.

Opis bota

Nakon citanja online manuala - napravio sam kod koji za 30 sek radi verifikaciju da je moguce pristupiti nalogu.

Sa kljucevima koji su procurili, okacio sam nekoliko tvitova.

Ownage slika je jednim dijelom ministrova

Takodje sam napomenuo ministra da ukloni kljuceve koje on ni do dan danas nije uspio da ukloni. Zapravo uklonio je privatne informacije sa bota tako da ne bi mogao biti povezan. Sem profilne slike koja je 50% njegova.

Takodje sam poslao nekoliko poruka DM tako da uklone kljuceve koje koriste njegovim pratiocima pa i ministru Rajcevicu.

Nakon toga je samo uklonio informacije sa opisa bota kao i link ka blogu. Removed info

Prije svega ovoga bio sam blokiran na tviteru od strane ministra nakon nekoliko pitanja za koje nije mogao il nije zelio da da odg. Pitanaj su bila uljudna i bez ruznih rijeci.

Block

Da sumiramo sve ovo: Kod ovog tviter bota je online 3 god i pored svih ljudi u ministarstvu, CERTu, i jos nekoliko kompanija sa kojima saradjuju nisu bili u stanju da uklone ovaj propust. Sada se postavlja pitanje sta je sa kompletnom infrastrukturom vlade RS i agencija. Naravno nisam imao ugovor sa ministrom niti vladom RS za ovo ali je kljuc bio tu svo vrijeme i pitanje je sta bi se desilo kada bi neko ozbiljan krenuo na kompletnu infrastrukturu vlade RS? Da li ovo znaci da nisu u stanju da zastite sopstveni kod a kamoli kompletne mreze i servere ? . Trebalo mi je nekoliko minuta da sve ovo uvezem - zamislite profesionalne grupe sajber kriminalaca.

Pouka svega je da nikada ne ostavljate u otvorenom tekstu sifre, kljuceve i druge bitne stvari - a pogotovo da i sifrovane ne leakujete na internet.

Ovo me podsjeca na likove koji su pravili kod u C za irc botove tamo 1999 do 2001 sa bugom tipa format string (ako neko zna o cemu se radi) i kako je bilo moguce preko nekoliko komandi da se odstampa sifra i user glavnog korisnika i pristupi botu. To su bila neka druga vremena.

EPILOG: nekoliko ljudi je naslo da su napravili istu gresku - pa su mi se zahvalili za pisanje clanka jer su i sami uklonili slicne probleme (pobrisane kljuceve, sifre iz koda koji su jos uvijek tu na githubu)

0wnage of twitter bot from local Bosnia and Herz/RS ministry of cybersecurity

In free time, I stumbled accidentally to Github repo of current minister for cybersecurity and computer science (and 2 other things that I don't remember).

here: github - Srdjan Rajcevic

After 3 min review - I found "delete" credentials from SQLite DB. My first thinking was that cred is used for private profile - so I report to local CERT RS on this. The response the same day as case closed.

CERT response

But it was not. Keys did not delete or revoked. After 24h I checked

After reviewing what keys are used - I found it that was meant to be part of ex-CERT twitter bot for news. But it was never finished. Here is ex blog of the minister blog webarchive

Bot description

I make simple bot code in less than 30 sec (just google it for API and documentation)

Run with proper keys. Boom! ownage twitter bot has a half picture of minister

Also, I send DM to all followers to revoke bot keys. But up today - keys are not revoked. Also, all info is removed I guess it was hard to receive this kind of info especially if you are marked as someone who can protect the complete infrastructure of one government. removed info from the bot

And before this, I got blocked from the account minister for "cybersecurity" and 3 things I don't remember. My question on some topic hit him hard.

block

To sum all this: The bot was there for 3 years and no one from ex and current CERT was able to catch "leaked credentials" which wonder me - what would happen if we start to review the complete infrastructure of government? Yes, I don't have a penetration-tester agreement so this was closed to "hack a profile" of the minister. It was fun - not a hacking just joke with ppl. Take me no more than 5 min (3 to find + 1 read a review of bot + 1 for the PoC)

Do not leave cred in the open text - always encrypt. Use some strong encryption to protect your secret. Then do not share your secret in any way. Review your code.

Also, this reminds me like the 1999-2001 period when beginners code in C for IRC bots and they leave format string bug on some command and you print complete memory and get password/user for master user. That was a good time.

EPILOG: several people thank me on this because they did same mistake.