In recent years video communication has occupied a large percentage of Internet traffic and the trend is expected to increase. Many services are offered via the Internet use video content to reach millions of users. Real time video services in the Internet are mostly done by using User Datagram Protocol (UDP). UDP is preferable because it is fast, but has a weakness of not supporting congestion control mechanisms; hence it usually floods the channel it is using causing congestion which then leads to packet loss, excessive delay and jitter. To support good quality of experience (QoE), packet loss, delay and jitter must be kept at accepted values.This paper presents part of an ongoing MSc. research work whose aim is to develop a rate adaptive congestion control scheme using a lookup table. The scheme use Real Time Transport Protocol (RTP) and RTP Control Protocol (RTCP) to provide end to end network transport service which is not provided by UDP and control service. The proposed congestion control scheme comprises of two components:- a lookup table and congestion control equation. The lookup table is created at the beginning of sender’s application and keeps a record of fraction loss and Inter Packet Gap (IPG), which is the interval in time between sending packets. Upon receiving a report the lookup table is searched to see if there is a match for the report’s fraction loss value and returns, the appropriate IPG value. If there is no match a congestion control equation is used to compute a new IPG value, which is then added to the lookup table and used for rate adjustment.Simulation of the proposed work was done using NS3, where by the Quality of Service (Qos) parameter which affect the Quality of Experience (QoE) such as delay and jitter were observed. Elvavid, a video quality evaluation tool is used to measure the quality of video by using Peak Signal Noise Ratio (PSNR) and Mean Opinion Score (MOS) measurements.