Record and Replay Keyboard & Mouse Activity Using Python

Imagine being able to record every mouse move and keypress you make, then replay them automatically—like having a personal assistant bot. With Python and a few handy libraries, this isn’t just possible… it’s actually pretty easy!

Here’s a real-world script example that allows you to record your mouse and keyboard activity, save it, and replay it later just like a macro recorder.

Example Script:

import pyautogui
import keyboard
import time
import json
from pynput import mouse
from threading import Thread

actions = []
recording = False
pressed_keys = set()  # ✅ Tambahan untuk mencegah duplikasi key

def record_action(event_type, details):
    timestamp = time.time()
    actions.append((timestamp, event_type, details))

def record():
    global recording, actions, pressed_keys
    actions = []
    pressed_keys = set()  # ✅ Reset state
    recording = True
    print("📼 Recording started. Press Ctrl+Shift+S to stop.")

    start_time = time.time()

    # ✅ Hook keyboard dengan pencegahan duplikasi
    def on_key_event(e):
        if not recording:
            return

        if e.event_type == 'down' and e.name not in pressed_keys:
            pressed_keys.add(e.name)
            record_action('key', e.name)
        elif e.event_type == 'up' and e.name in pressed_keys:
            pressed_keys.remove(e.name)

    keyboard.hook(on_key_event)

    # Hook mouse events
    def on_click(x, y, button, pressed):
        if recording:
            if pressed:
                record_action('mousedown', (x, y, str(button)))
            else:
                record_action('mouseup', (x, y, str(button)))

    def on_move(x, y):
        if recording:
            record_action('move', (x, y))

    mouse_listener = mouse.Listener(on_click=on_click, on_move=on_move)
    mouse_listener.start()

    # Stop logic
    while recording:
        if keyboard.is_pressed('ctrl+alt+shift+s'):
            recording = False
            print("🛑 Recording stopped.")
            break
        time.sleep(0.01)

    mouse_listener.stop()

    # Save actions with relative timestamp
    base_time = actions[0][0] if actions else time.time()
    relative_actions = [(round(t - base_time, 4), tpe, data) for t, tpe, data in actions]

    with open("actions.json", "w") as f:
        json.dump(relative_actions, f)

    print(f"💾 Saved {len(relative_actions)} actions to actions.json")

def replay():
    speed_multiplier = 50.0  # 🔥 bisa diubah misalnya 10.0 untuk turbo

    try:
        with open("actions.json", "r") as f:
            recorded = json.load(f)
    except:
        print("❌ actions.json not found.")
        return

    print(f"▶ Replaying actions {speed_multiplier}x faster...")

    for i, (delay, event_type, details) in enumerate(recorded):
        if i > 0:
            prev_delay = recorded[i - 1][0]
            delay_diff = (delay - prev_delay) / speed_multiplier
            time.sleep(max(0, delay_diff))

        if event_type == "move":
            x, y = details
            pyautogui.moveTo(x, y)
        elif event_type == "mousedown":
            x, y, button = details
            pyautogui.mouseDown(x, y, button=button.replace("Button.", ""))
        elif event_type == "mouseup":
            x, y, button = details
            pyautogui.mouseUp(x, y, button=button.replace("Button.", ""))
        elif event_type == "key":
            pyautogui.press(details)

    print("✅ Done replay!")

if __name__ == "__main__":
    print("=== KESO RECORDER ===")
    while True:
        cmd = input("Type 's' to start recording, 'doit' to replay, or 'exit': ").strip().lower()
        if cmd == 's':
            record()
        elif cmd == 'doit':
            replay()
        elif cmd == 'exit':
            break
        else:
            print("❓ Unknown command.")

What Is This Script For?

This Python script lets you:

• Record your interactions with your computer, including clicks, mouse movements, and keyboard inputs.
• Save those interactions to a file called actions.json.
• Replay those actions automatically, mimicking your behavior in real-time (or faster!).

It’s great for:

• Automating repetitive tasks,
• Creating software usage demos,
• UI testing,
• Or just playing around with automation.

How Does It Work?

The script has two main features:

1. record(): Recording Your Activity

• Uses keyboard.hook() to track key presses and releases, with duplicate prevention so it doesn’t keep logging a key being held down.
• Uses pynput.mouse.Listener to track mouse movements and clicks.
• Every event is stored as (timestamp, event_type, event_detail).
• Press Ctrl + Alt + Shift + S to stop recording.
• Events are saved in a JSON file with relative timestamps, making it easy to speed up or slow down the replay.

2. replay(): Playing It Back

• Reads from the actions.json file.
• Uses pyautogui to simulate the same actions, such as:
  • Moving the mouse,
  • Clicking,
  • Pressing keys.
• Playback speed is controlled with speed_multiplier, e.g., 50.0 makes it 50x faster than real time.

How to Use It

1. Run the script in your terminal or command prompt.
2. At the prompt:
   • Type 's' to start recording,
   • Type 'doit' to replay,
   • Type 'exit' to quit.
3. While recording, interact with your apps as usual.
4. Press Ctrl + Alt + Shift + S to stop recording.
5. Use the replay command to watch your actions play back automatically.

Summary

This script is a great example of how Python can be used to build real, practical automation tools. With a bit of creativity, you can turn this into:

• A personal macro recorder,
• A UI testing helper,
• Or even a basic RPA (Robotic Process Automation) system.

Give it a shot, tweak it, and explore the world of Python automation! 🐍⚙️