I have made this dive from the beach and from a boat. The bottom is rocky with boulders and a fast moving current. To do this from the beach you need to climb down a huge stairway to the beach, 120 steps in all. (See info link for picture of steps) From the beach this site needs to be timed at high slack tide. I made this dive from a boat during an outgoing tide and it was like an underwater roller coaster taking you over boulders as big as a two story house.
From Aqua Explorers:
Take the Long Island Expressway east to Exit 73, Old Country Road. Drive east on Old Country Road; it will change into Main Road. Once into the town of Southold, turn left onto Youngs Avenue. Drive north on Youngs Avenue to the end. Turn right onto North Road, then a quick left on Lighthouse Road.
Lighthouse Road will bring you into a parking lot that overlooks Horton Point Beach. The parking lot is posted "No Parking l0 p.m. to 7a.m."