FeaturedLive
CoSigning
A two-way translation platform between American Sign Language (ASL) and speech.
The problem
Deaf and hard-of-hearing people struggle to communicate day to day with people who don't know sign language.
The solution
Speech to sign: the site captures a spoken sentence, turns it into text, and plays a video for each sign in order. Sign to speech: a deep-learning Transformer model analyses webcam or uploaded video, extracting hand and body movement with MediaPipe and visual features with ViT, then turns the signs into an English sentence that is read aloud.
My role
A TÜBİTAK 2209-A supported project. I developed the platform end to end and reached about 94% sign recognition accuracy.
- Python
- PyTorch
- Vision Transformer (ViT)
- MediaPipe
- OpenCV
- FastAPI
- JavaScript
- Web Speech API
- HTML
- CSS
